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SUMMARY 

The  objectives  of  this  study  were  to  design,  develop,  implement  and 
evaluate  a  videodisc-based  simulator  system,  the  Interactive  Graphics 
Simulator  (IGS)  for  6883  Converter  Flight  Control  Test  Station  training 
at  Lowry  AFB,  Colorado.  The  IGS  was  more  than  a  functional  replication 
of  the  actual  equipment,  it  also  specifically  covered  maintenance 
planning  skills  (i.e.,  decision  making  and  problem  solving)  within  the 
simulations.  Setting  controls  and  accessing  test  points  on  the  IGS  was 
done  using  a  touch  screen  or  touch  pad  to  select  areas  of  a 
video-disc-based  visual. 

The  students  in  the  training  course  (N=43)  were  assigned  to  either 
an  actual  equipment  trainer  (AET)  or  an  IGS  training  mode.  AET-trained 
students  were  given  hands-on  equipment  manipulation  training  on  the  6883 
test  station  while  IGS-trained  students  received  equipment  manipulation 
and  troubleshooting  training  on  the  IGS  system.  Student  performance  was 
assessed  using  a  battery  of  tests  designed  to  measure  troubleshooting 
ability,  equipment  operational  procedures  performance,  and  field 
assignment  readiness.  These  scores  were  supplemented  with  conventional 
end-of-block  test  results  and  student  attitudinal  data.  Costs  of  the 
IGS  system  and  the  AET  were  also  compared. 

Analysis  of  the  scores  showed  the  IGS  system  to  be  as  effective  as 
the  actual  equipment  in  training  hands-on  procedures  and  to  be  superior 
to  the  AET  in  training  troubleshooting  skills.  The  IGS  also  proved  to 
be  suitable  for  the  training  environment,  providing  for  easy  courseware 
development,  requiring  minimal  maintenance,  and  operating  efficiently  in 
a  standard  classroom.  Student  acceptance  of  the  IGS  system  was  high. 
The  IGS  system  not  only  provided  training  effectiveness  advantages,  but 
also  substantial  cost  savings  over  the  AET. 


PREFACE 


This  report  is  the  final  report  for  the  design,  development, 
implementation,  and  evaluation  of  the  Interactive  Graphics  Simulator 
system  for  maintenance  training.  The  project  was  conducted  for  the  Air 
Force  Human  Resources  Laboratory  Technical  Training  Division,  Lowry  AFB, 
Colorado.  This  effort  is  documented  under  Project  2361  -  Simulation  for 
Maintenance  Training.  The  goal  for  this  effort  was  to  utilize  a 
computer-driven  videodisc  based  training  simulator  in  place  of  the 
actual  equipment  trainer  normally  used  in  one  instructional  block  of  the 
F-lll  Integrated  Avionics  Computerized  Test  Station  and  Component 
Specialist  training  course. 

This  effort  was  conducted  from  April  1981  through  August  1984.  The 
services  of  Essex  Corporation  were  obtained  through  contract  AF33615-81- 
C-0006  for  which  Mr.  William  J.  Pieper  was  the  Principal  Investigator 
and  Project  Director.  Collaborative  project  effort  was  obtained  through 
a  subcontract  with  the  Denver  Research  Institute  for  which  Dr.  J. 
Jeffrey  Richardson  was  the  Project  Director.  The  Air  Force  contract 
technical  monitor  was  Capt  Randy  H.  Massey. 
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I.  INTRODUCTION 
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Background 

Based  on  a  recent  study  of  simulators  for  maintenance  training,  a 
paraphrased  definition  is:  training  devices  that  replicate  many,  but  not 
necessarily  all,  of  the  appearances  and  functions  of  actual  equipment 
(Orlansky  &  String,  1983).  Based  on  their  external  physical  appearance, 
they  are  often  characterized  as  two-  or  three-dimensional  devices.  The 
three-dimensional  simulator  bears  the  closest  resemblance  in  physical 
form  and  function  to  the  actual  equipment.  The  two-dimensional 
simulator  has  less  physical  realism  and  represents  the  actual  equipment 
through  pictures  presented  on  flat  panels,  augmented  with  selected 
functioning  controls,  displays,  and  test  points.  Although  differing  in 
the  extent  of  realism,  current  maintenance  training  simulators  attempt 
to  replicate  the  physical  appearance  and  functions  of  the  actual 
equipment;  they  are  equipment  simulators.  In  general,  the  major 
advantages  of  simulators  over  actual  equipment  for  maintenance  training 
include  increased  reliability,  improved  safety,  greater  opportunity  for 
troubleshooting  practice,  and  reduced  cost. 

Several  studies  comparing  training  simulators  and  actual  equi1' 
trainers  (AETs)  were  reviewed  to  determine  the  general  finding'  .ch 
regard  to  training  effectiveness  and  cost  (Orlansky  &  String,  198  In 

general,  the  studies  indicated  that  equipment  simulators  could  be  used 
for  training  in  place  of  AETs  at  substantial  cost  savings  wiv 
sacrificing  training  effectiveness.  At  least  two  recent  studies 

corroborated  these  findings  (Cicchinelli,  Harmon,  &  Keller,  1982; 
Cicchinelli,  Keller,  &  Harmon,  in  press).  Taken  strictly  as 

representations  of  actual  equipment,  equipment  simulators  used  in 
maintenance  training  have  done  as  well  as  can  be  expected,  for  they 
could  not  be  expected  to  train  any  better  than  the  actual  equipment. 

A  newer  generation  of  simulators,  called  here  an  Interactive 
Graphics  Simulator (IGS) ,  bears  less  physical  resemblance  to  the  actual 
equipment  than  either  the  two-  or  three-dimensional  equipment 
simulators.  An  IGS  represents  the  equipment  layout,  controls,  and 
displays  pictorially  on  a  color  video  monitor  using  videodisc  images  as 
the  source.  Setting  controls  and  accessing  test  points  on  the  video 
monitor  are  done  through  the  use  of  touch  panels,  light  pens,  or  similar 
electronic  selection  devices.  The  challenge  for  a  maintenance  training 
IGS  was  to  achieve  a  training  effectiveness  advantage  over  equipment 
simulators  or  actual  equipment  trainers.  To  do  this,  the  IGS  had  to  be 
more  than  a  representation  of  the  actual  equipment.  Specifically,  it 
had  to  be  a  device  which  aided,  with  evaluation,  the  learner's  emulation 
of  maintenance  tasks  and  skills;  as,  for  example,  diagnosing 
malfunctions;  identifying,  repairing,  and  replacing  parts;  and  operating 
the  equipment  to  verify  proper  functioning. 

Achieving  increased  training  effectiveness  was  taken  to  mean  the 
incorporation  of  some  form  of  computer-based  instructional  (CBI) 
technology;  that  is,  the  capability  to  prompt,  evaluate,  and  remediate 
maintenance  skills.  The  definition  of  maintenance  simulation  was 
broadened  to  include  not  only  equipment  representation  but  also  task 
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emulation.  The  definition  of  an  IGS  for  training  was  extended  to:  "A 
training  device  that  (a)  requires  the  user  to  emulate  the  tasks,  skills, 
and  knowledges  of  a  trained  maintainer  and  (b)  simulates  the  physical 
appearances  and  functions  of  actual  equipment."  This  orientation,  along 
with  the  implied  computer-based  system,  was  anticipated  to  achieve  the 
training  effectiveness  advantages  desired  and  at  the  same  time  maintain 
the  cost  advantages  expected  of  equipment  simulators. 


Pro iect  Goals 

The  goals  of  this  project  were  to  design,  develop,  implement,  and 
evaluate  an  interactive  computer-based  IGS  in  support  of  avionics 
maintenance  training.  The  required  IGS  system  was  to  provide  for  the 
use  of  task  analysis  data,  as  well  as  equipment  manipulation  data.  The 
task  analysis  data  base  was  designed  to  be  built  on-line,  representing 
all  of  the  trained  maintenance  technician  skills  and  knowledges.  The 
task  data  base  was  also  designed  to  serve  simultaneously  as  the  training 
data  base  and  as  the  student  model.  The  equipment  manipulation  data 
base  was  designed  to  be  built  on-line  using  videodisc  picture 
information  and  digitized  graphic  overlays  for  presentation  with  the 
videodisc  images. 

During  development,  a  set  of  program  tools  was  built  for  use  in 
implementing  the  graphics  simulations.  The  tools  captured  a  methodology 
of  courseware  development.  The  tools  consisted  of  editors  for  the  task 
and  equipment  manipulation  data  bases,  a  text  and  graphics  editor  for 
the  task  data  base,  and  a  graphics  editor  for  the  equipment  data  bases. 
In  addition,  run-time  programs  were  built  for  the  task  and  equipment 
manipulation  elements  that  interfaced  the  data  bases  and  the  student 
model.  The  approach  of  using  program  tools  as  development  aids  provided 
an  efficient  means  of  courseware  development,  lessening  the  acknowledged 
labor-intensive  nature  of  CBI  materials  development. 


II.  INTERACTIVE  GRAPHICS  SIMULATOR  DESIGN  AND  DEVELOPMENT 


Before  the  detailed  design  of  the  IGS  is  presented,  the  simulator 
will  be  described  from  the  perspective  of  a  trainee.  The  trainee  sits 
before  a  keyboard,  a  touch-sensitive  pad,  and  two  displays.  The  video 
monitor  presents  videodisc  images  with  graphic  overlays  providing  actual 
equipment  static  and  motion  display.  User  interaction  with  the  monitor 
is  accomplished  through  the  touch  pad  which  provides  selection  of 
desired  positions  on  the  display.  The  terminal  presents  the 
problem-solving  components  of  the  tasks.  User  interaction  with  the 
terminal  is  accomplished  through  the  keyboard. 

Trainee  interaction  begins  using  the  terminal  display  of  the  IGS. 
The  terminal  uses  a  medium-resolution  graphics  display  on  which  text, 
diagrams,  decision  alternatives  (menus),  prompts,  and  feedback  messages 
are  presented.  For  example,  suppose  a  trainee  is  developing  a  block 
diagram  of  the  signal  path  involved  in  a  troubleshooting  problem. 
Figure  1  depicts  one  step  in  this  process.  At  this  point,  the  trainee 
will  be  focused  on  the  inputs  and  outputs  for  one  particular  functional 
unit,  the  PSC  (phase  sensitive  converter).  The  trainee  has  already 
identified  that  the  input  should  be  50  volts  DC  (VDC)  and  has  two  more 
signals  to  identify.  The  terminal  display  shows  a  graphic  representing 
the  PSC  with  its  two  inputs  and  one  output.  The  signal(s)  already 
identified  (in  this  case  the  50-VDC  input)  is  labeled  and  highlighted. 
A  text  prompt  beneath  the  graphic  asks,  "Which  signal  would  you  like  to 
add  to  the  block  diagram  now?"  At  the  bottom  of  the  task  display,  the 
decision  alternatives  to  be  used  in  answering  this  question  are 
presented.  At  this  point  in  the  example,  the  correct  response  is  either 
"Reference"  or  "Output."  Feedback  messages  associated  with  each 
incorrect  response  are  displayed  if  that  response  is  chosen.  For 
example,  if  the  trainee  chooses  "Input,"  the  displayed  feedback  message 
is,  "You  have  already  identified  this  signal,  pick  another!" 
Interaction  on  the  task  display  continues  until  an  equipment  action  is 
required  (e.g.,  Set  up  voltmeter).  At  that  time,  control  is  passed  to 
the  equipment  display  monitor. 

The  equipment  display  monitor  is  a  regular  TV  color  video  monitor. 
This  monitor  is  used  in  a  fashion  that  has  become  almost  standard  in 
microform  and  videodisc  simulations  of  equipment.  The  student  accesses 
controls,  readouts,  test  points,  and  components  by  selecting,  through 
the  touch  pad,  the  portion  of  the  picture  to  be  displayed  in  greater 
detail.  For  example,  if  a  voltmeter  range  control  is  to  be  set  from  100 
to  10,  the  trainee  begins  by  selecting  the  voltmeter  drawer  in  a  rack  of 
equipment  pictured  on  the  monitor.  The  rack  picture  is  replaced  by  a 
picture  of  the  voltmeter  drawer.  Next,  the  range  control  is  selected 
and  the  voltmeter  picture  is  replaced  by  a  picture  of  the  range  control, 
currently  positioned  at  100.  At  this  point,  the  10  on  the  range  scale 
is  selected,  and  a  picture  of  the  range  control  set  to  10  is  displayed. 
Finally,  when  the  equipment  actions  are  complete,  the  trainee  selects  an 
area  on  the  monitor  screen  labeled  "Done." 

When  the  voltmeter  is  correctly  set  up,  control  is  again  returned 
to  the  terminal  display.  In  this  way,  a  simulation  proceeds  through  an 
initial  sequence  of  task  displays,  setting  the  problem  scenario  to  the 


Which  signal  would  you  like  to 
odd  to  the  block  diagram  now  ? 

INPUT?  REFERENCE?  OUTPUT? 

•  •  • 


Depicted  is  an  intermediate  step  in  the  development  of  a  block 
diagram.  Here  the  trainee  has  already  identified  that  the  INPUT  shouid  be  50 
VDC  and  has  two  more  signals  to  identify. 


Figure  1.  Typical  task  display. 
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first  equipment  action.  Student  activity  continues  through  simulated 
equipment  actions  on  the  equipment  display  monitor  and  then  returns  to 
the  next  series  of  problem-solving  activities  on  the  terminal  display. 
This  process  continues,  alternating  between  the  two  instructional 
displays,  until  the  problem  is  completed.  The  presented  simulations  are 
task-centered,  in  that  the  planning  and  problem-solving  activities 
control  access  to  the  equipment  display  monitor. 

IGS  System  Design 

Quality  in  the  development  of  task-centered  simulations  is  impacted 
by  (a)  the  accuracy  and  completeness  of  decision  alternatives,  (b)  the 
consistency  in  problem-solving  approach  from  simulation  to  simulation  as 
reflected  in  the  decision  alternatives,  (c)  the  cost  incurred  in 
developing  comprehensive  sets  of  decision  alternatives  for  an  entire  set 
of  simulations,  and  (d)  the  ability  to  chain  together  equipment  actions 
that  should  be  performed  as  a  single,  integrated  equipment  sequence 
given  sufficient  trainee  expertise.  The  IGS  system  incorporated  the 
development  and  use  of  an  on-line  task  analysis  technique  in  order  to 
address  these  concerns. 

On-Line  Task  Analysis 

The  developed  on-line  task  analysis  technique  was  defined  as.  the 
generation  of  a  computer-resident  task  reduction  data  base  containing 
the  set  of  cognitive  and  manipulative  actions  required  to  accomplish  a 
task.  The  technique  captured  the  problem-solving  strategy  known  as 
hierarchical  decomposition  or  planning  by  abstraction.  For  any  task, 
the  strategy  was  to  divide  the  task  into  more  and  more  simple  subtasks. 

The  approach  taken  in  this  task  analytic  technique  was  to 
continually  ask,  "What  must  be  done  to  accomplish  the  task?"  If  three 
things  had  to  be  done  (A,  B,  and  C),  the  question  was  repeated  for  each 
of  these.  "What  must  be  done  to  accomplish  A?"  The  process  was  repeated 
until  the  answer  to  "What  must  be  done?"  was  assumed  to  be  part  of  the 
repertoire  of  the  trainee  or  was  in  a  sense  "elemental."  For  example, 
changing  the  range  on  a  voltmeter  from  100  to  10  was  considered 
elemental;  i.e.,  the  smallest  level  of  behavior  to  be  tracked  in  the 
system. 

The  nature  of  the  relationship  between  the  task  and  the  equipment 
was  neatly  represented  by  the  task  analysis.  The  outcome  of  the  task 
analysis  had  the  form  of  a  tree.  The  top  or  root  node  of  the  tree 
represented  the  most  global  description  of  the  task.  Several  branches 
emanated  from  the  root  node  to  subordinate  nodes  representing  the  major 
subtasks.  Each  of  the  subtasks  had  branches  emanating  to  nodes 
representing  their  subparts,  and  so  on,  until  the  task  was  decomposed 
into  elemental  equipment  actions.  Therefore,  all  equipment  actions  were 
terminal  nodes  in  the  tree. 

The  target  task  for  IGS  system  development  was  the  operation  and 
maintenance  of  an  6883  Test  Station  (Dallman,  Pieper,  &  Richardson, 
1983).  Figure  2  provides  an  abbreviated  listing  of  the  outcome  of  the 
analysis  for  this  task.  The  full  analysis  tree  is  up  to  16  levels  deep 
and  contains  well  over  1,000  nodes. 
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Only  one  line  of  decisions/actions  is  represented;  all  others  have  been 
pruned,  as  indicated  by  the  ellipses.  Task  elements  listed  in  italics  are  terminal 
nodes  and  entail  equipment  manipulation. 

Figure  2.  Task  analysis  tree  for  6883  maintenance. 
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Detailed  IGS  Design  Goals 

Four  detailed  IGS  design  goals  were  addressed  in  the  development  of 
the  on-line  task  analysis  technique. 

Design  Goal  1 :  Focus  the  simulation  on  the  tasks .  not  the 
equipment .  The  inherent  structure  of  the  developed  on-line  task 
analysis  technique  made  equipment  simulation  an  integral  but  subordinate 
portion  of  the  decomposed  task.  This  is  graphically  depicted  by  the 
fact  that  all  equipment  actions  were  terminal  nodes  in  the  resultant 
task  analysis  tree.  Given  this  goal,  the  IGS  was  clearly  defined  as  a 
task  emulation  device,  not  merely  an  equipment  simulator. 

Representation  of  the  equipment  was  still  a  vital  aspect  of  this 
approach  to  maintenance  simulation.  There  had  to  be  some  way  to 
practice  equipment  actions.  Therefore,  the  videodisc  representation  of 
the  equipment  was  a  substantial  element  of  the  simulator. 

Design  Goal  2 .  Use  the  task  data  base  to  develop  simulation 
courseware  through  an  authoring  editor .  An  authoring  editor  was 
designed  to  flag  a  correct  path  through  the  task  analysis  tree  as  the 
means  of  developing  each  unit  of  computer-based  simulation  courseware 
(i.e.,  problem  protocol).  The  editor  was  built  to  do  this  by  displaying 
the  current  node  of  the  task  tree  along  with  each  of  its  children  (i.e., 
subordinate  nodes).  A  subject-matter  expert  (SME)  was  to  identify  which 
child  or  children  must  be  accomplished  at  this  level  of  the  tree  for 
this  protocol  and  in  what  order  each  must  be  done.  The  SME  then 
selected  one  of  the  children  as  the  new  current  node  and  the  process  was 
repeated.  Through  this  mechanism,  the  task  analysis  tree  was  caused  to 
serve  directly  as  the  basis  for  development  of  the  simulation  courseware 
which  trains  the  job  tasks. 

Simulation  protocols  were  built  with  the  editor  as  follows.  As  the 
SME  selected  the  subordinate  nodes  (children)  to  be  performed  at  each 
level  of  the  tree,  the  order  of  accomplishment  was  also  determined.  For 
example,  the  selected  children  could  be  assigned  to  be  performed 
serially;  that  is,  do  A,  then  do  B,  then  do  C,  etc.  Alternatively,  they 
could  be  assigned  in  various  logical  combinations  of  order;  e.g.,  first 
do  either  A  or  C,  then  do  B,  and  finally,  do  D  and  E  in  any  order. 
Figure  3  illustrates  the  appearance  of  the  editor  screen  after  order 
assignment  for  the  children  of  the  "CHECK  PATCH  PANEL"  node  of  the  task 
data  tree. 

Once  order  was  assigned  for  all  of  the  nodes  to  be  accomplished  in 
a  protocol,  the  task  display  associated  with  each  decision  alternative 
(e.g.,  first  do  A  or  C)  was  developed.  Recall  that  the  task  display  is 
what  is  presented  to  the  trainee  when  that  decision  alternative  is 
pending  (see  Figure  1).  The  SME  would  use  graphics  and  text  editors  to 
develop  the  task  displays  which  contained  text  and  graphics  summarizing 
the  current  state  of  the  problem  and/or  presenting  comment  or  prompts 
about  the  pending  decision.  Finally,  textual  feedback  messages  were 
created  by  the  SME  for  each  of  the  incorrect  choices  for  the  decision 
pending  when  the  task  display  was  presented.  If,  for  example,  the 
pending  decision  were  "do  either  A  or  C,"  feedback  messages  would  be 
prepared  for  choices  B,  D,  and  E.  No  feedback  would  be  prepared  for  A 
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CHECK  PATCH  PANEL  Subtosks= 

TASK  ACTIONS  TO  BE 
PERFORMED^ 

«  Install  cover 

1.  Install  cover 

*  Set  up  measurement 

2.  Set  up  measurement 

device 

device 

*  Enter  program 

3.  Select  pins 

*  Select  pins 

4.  Enter  program 

5.  Enter  program 

—  menu  of  editor  options  normolly  oppeors  here  — 

< _ > 

This  display  corresponds  to  the  task  structure  depicted  in  Figure  2. 
Suppose  "Check  Patch  Panel"  was  the  method  selected  for  checking  the  switching 
signals  of  TPS  Group  II  relays  (refer  to  Figure  2).  When  the  "Check  Patch  Panel" 
node  is  expanded,  the  above  display  appears  with  the  task  components  listed  on 
the  left  and  a  blank  screen  on  the  right.  The  SME,  using  editor  commands 
normally  listed  at  the  bottom  of  the  screen,  assigns  the  task  components  required 
to  perform  the  patch  panel  check  for  this  problem.  The  result  is  the  list  of  task 
components  shown  on  the  right-hand  side  of  the  screen.  At  this  point,  if  the  SME 
entered  a  "2,"  this  would  cause  the  "Set  Up  Measurement  Device"  node  to  be 
expanded  and  the  process  of  simulation  courseware  development  would  continue. 
Task  displays  are  created  by  entering  a  command  to  "create  task  display"  and  the 
number  (from  1  to  5  in  this  example)  of  the  task  action  to  associate  with  the 
display.  Feedback  messages  are  similarly  constructed.  The  menu  the  trainee  sees 
at  the  bottom  of  a  task  display  is  retrieved  automatically  from  the  task  analysis 
tree  and  consists  of  all  items  listed  on  the  left-hand  half  of  the  editor  screen. 

Figure  3.  Simulation  courseware  development  editor. 
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and  C  because  these  would  be  correct  decision  alternatives.  A  task 
display  was  considered  complete  when  it  contained  the  text  and  graphics 
for  the  pending  decision  and  the  feedback  messages  for  the  incorrect 
decision  alternatives.  Once  developed,  the  task  displays  were  stored  as 
files  until  protocol  run  time. 

The  situation  with  regard  to  terminal  nodes  (i.e.,  equipment 
actions)  was  slightly  different.  Task  displays  were  prepared  for  use 
when  the  equipment  nodes  were  the  pending  decision  alternatives.  In 
addition,  sequences  of  videodisc  images  and  lists  of  touch  pad  inputs 
were  also  prepared  for  the  actual  equipment  simulation  components.  The 
equipment  action  files  (i.e.,  step  files)  were  stored  separately  but 
were  referenced  to  the  node  in  the  task  data  tree  for  which  they  were 
relevant.  The  example,  setting  the  voltmeter  range  to  10,  given  near 
the  beginning  of  this  report,  would  require  that  a  sequence  of  videodisc 
images  and  touch  pad  inputs  be  set  up  corresponding  to  the  following 
actions:  select  the  voltmeter  in  the  rack  of  equipment,  select  the  range 
control  on  the  pictured  face  of  the  voltmeter,  select  the  area  labeled 
"10"  on  the  picture  of  the  range  control,  and  select  the  "Done"  area  at 
the  top  of  the  display  screen. 

In  summary,  the  task  analysis  tree  provided  a  ready  means  of 
developing  simulations.  The  decision  alternatives  for  each  step  were 
provided  directly  by  the  data  base.  The  SME  identified  the  correct  path 
through  the  tree  for  the  protocol  at  hand  and  then  associated  task  or 
equipment  display  information  with  each. 

Design  Goal  3 :  Promote  consistency  in  the  problem-solving  approach 
while  allowing  for  flexibility  in  simulation  design.  Since  all 
simulations  were  developed  from  the  same  task  analysis  tree,  they 
presented  a  consistent  problem-solving  approach.  That  is,  any  menu 
(list  of  decision  alternatives)  displayed  in  one  simulation  was  exactly 
the  same  in  all  other  simulations  which  needed  to  display  that  menu.  The 
menus  were  not  entered  in  the  task  display  by  the  SME;  they  were 
provided  by  the  task  analysis  tree.  Only  the  correct  choices  for  the 
specific  simulation  were  provided  by  the  SME.  Consistency  in  the 
approach  from  simulation  to  simulation  was  inherent  in  the  use  of  the 
task  data  tree  for  providing  both  the  decision  alternatives  to  the  SME 
and  menu  choices  to  the  trainees. 

This  approach  to  authoring  supported  a  wide  degree  of  latitude  in 
constructing  simulation  problems.  In  particular,  it  supported  the 
development  of  part  tasks  and  the  development  of  alternative 
troubleshooting  approaches.  An  example  will  illustrate  one  way  in  which 
part  tasks  were  developed.  Suppose  decision  alternatives  at  some  point 
in  the  simulation  included:  complete  block  diagram,  perform  visual 
inspection,  and  check  signals.  It  may  be  that  the  SMEs  wished  at  this 
time  to  focus  on  the  check  signals  component  of  the  task  and  to  pass 
over  completing  the  block  diagram  and  performing  a  visual  inspection. 

In  order  to  accomplish  this,  the  SME  might  have  assigned  all  three 
subnodes  in  sequence.  At  the  same  time,  the  detail  correct  alternative 
paths  beneath  the  "Complete  Block  Diagram"  and  "Perform  Visual 
Inspection"  nodes  would  not  be  completed.  Instead,  task  displays  for 
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each  of  these  two  elements  would  have  been  created  summarizing  what  the 
trainee  would  have  accomplished  had  these  subtasks  been  completed.  Only 
the  path  beneath  the  "Check  Signals"  alternative  would  have  been  traced 
out  completely.  In  this  case,  the  scenario  for  the  trainee  would  have 
gone  as  follows:  "What  would  you  like  to  do  next — Complete  Block 
Diagram,  Perform  Visual  Inspection,  or  Check  Signals?"  Suppose  the 
trainee  had  correctly  selected,  "Complete  Block  Diagram."  In  response 
to  this,  the  trainee  would  have  seen  a  summary  task  display  showing  a 
completed  block  diagram  and  a  statement,  "Here  is  your  completed  block 
diagram.  Press  NEXT  to  continue."  The  trainee  would  then  be  returned 
to  the  menu  of  decision  alternatives:  "Complete  Block  Diagram,  Perform 
Visual  Inspection,  or  Check  Signals."  Next,  correctly  selecting 

"Perform  Visual  Inspection"  would  have  provided  a  summary  task  display 
of  what  that  task  would  have  yielded  at  its  completion.  The  trainee 
would  again  have  been  returned  to  the  menu:  "Complete  Block  Diagram, 
Perform  Visual  Inspection,  or  Check  Signals."  Finally,  correctly 
picking  "Check  Signals,"  would  have  provided  a  regular  task  display  with 
a  new  set  of  decision  alternatives  appropriate  to  performing  the  next 
task  in  the  decomposition  of  "Check  Signals." 

The  authoring  approach  also  supported  the  development  of 

alternative  troubleshooting  strategies.  For  example,  suppose  any  of 
three  functional  areas,  X,  Y,  or  Z ,  was  to  be  explored  next.  Further, 
suppose  the  fault  lies  in  region  X.  The  author  could  have  assigned 
these  areas  in  at  least  two  ways:  X,  Y,  and  Z  may  be  performed  in  any 
order,  or  Y  and  Z  are  optional,  and  X  is  required.  In  the  first  of 
these  assignments,  the  trainee  would  have  had  to  explore  all  three  areas 
even  if  X  had  been  done  first.  In  the  second,  once  area  X  had  been 
explored,  neither  of  the  others  would  have  to  be  done.  As  a  further 
example,  suppose  either  of  two  methods  for  checking  a  certain  signal  was 
acceptable.  Suppose  that  one  method  was  the  favorite  of  one  SME  and 

that  the  other  was  the  favorite  of  another  SME.  If  both  methods  were  to 

be  developed  in  detail,  they  could  have  been  assigned  as  follows,  "Do 
either  Method  A  or  Method  B."  The  task  analysis  subtrees  for  both 
Method  A  and  Method  B  would  have  been  developed  beneath  their  respective 
nodes.  If  development  resources  had  not  permitted  building  paths  for 
both  methods,  and  it  was  still  perceived  to  be  important  to  give  the 
trainee  the  option  of  selecting  either  method,  a  summary  scene  could 
have  been  developed  for  Method  B  stating,  "Yes,  you  could  use  Method  B. 
It  is  a  perfectly  legitimate  approach.  However,  today  we  would  prefer 
you  to  try  Method  A." 

The  use  of  the  task  data  tree  for  generation  of  the  decision 
alternatives  provided  the  desired  consistency  in  problem-solving 
approach.  In  addition,  the  authoring  editor  with  the  task  displays 
provided  the  desired  flexibility  in  simulation  design.  Both  of  these 
goals  were  achieved  simultaneously. 

Design  Goal  4 :  Adapt  the  leve 1  of  required  interaction  according 
to  trainee  achievement .  As  a  task  was  mastered,  it  became  appropriate 
to  support  the  chaining  of  common  equipment  actions.  For  example, 
consider  the  non-repertoire  task  of  setting  controls  on  a  voltmeter  to 
make  a  certain  measurement.  Assume  the  voltmeter  had  10  controls  and 
that  three  of  these  had  to  be  set  to  make  the  measurement . ^ i nr e  the  tns 
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was  not  in  the  repertoire  of  the  trainee,  the  task  of  setting  the 
controls  was  represented  in  the  task  data  base.  That  is,  each  control 
was  treated  as  a  separate  equipment  action  and  each  control  had  to  be 
selected  explicitly  on  the  task  display  before  it  could  be  set  on  the 
equipment  display  (see  Figure  4). 

After  an  adequate  amount  of  practice,  the  task  became  familiar  and 
it  was  no  longer  appropriate  to  force  the  trainee  to  plan  separately  for 
and  then  to  make  each  control  setting.  It  was  appropriate  to  allow  the 
trainee  to  select  the  goal  "Set  Voltmeter  Controls"  and  then  to  go 
directly  to  the  equipment  display  and  make,  in  an  integrated  fashion, 
each  of  the  control  settings  needed.  In  other  words,  the  level  of 
detail  or  graininess  of  the  simulation  had  to  be  adapted  to  the 
trainee's  level  of  competence.  This  was  achieved  through  the  use  of  a 
student  model  and  run-time  adaptation  rules. 

An  individualized  student  model  was  built  from  a  copy  of  the 
task  names  in  the  on-line  task  analysis  data  base;  one  for  each  trainee. 
The  model  was  designed  to  be  annotated  with  a  record  of  trainee 
performance  on  each  node  and  an  indication  of  the  current  state  of  the 
node.  After  the  trainee  had  correctly  accomplished  a  node  a  given 
number  of  times,  the  state  value  for  the  node  was  increased.  For  nodes 
with  task  displays,  the  state  values  were  specified  as  recognition, 
recall,  performance,  and  summary.  The  run-time  adaptation  rules  treated 
each  of  the  nodes  differently.  For  nodes  in  recognition,  the  menu  was 
presented  on  the  task  display  (i.e.,  decision  alternatives  were 
explicit).  In  recall,  the  menu  was  not  displayed  (i.e.,  alternatives 
had  to  be  retrieved  from  memory) .  A  node  in  the  performance  state 
caused  all  subordinate  task  display  nodes  to  be  ignored,  while  the 
subordinate  terminal  nodes  (the  equipment  actions)  had  to  be 
accomplished.  Once  a  node  reached  the  summary  state,  no  further  actions 
needed  to  be  performed  by  the  trainee;  the  results  of  the  actions  which 
would  have  been  required  were  summarized. 

In  the  example  above,  each  time  a  trainee  set  a  voltmeter  control, 
performance  on  that  task  was  recorded.  Once  the  task  was  performed 
correctly  a  given  number  of  times,  the  state  of  the  control's  node  was 
changed  and  the  task  display  changed.  At  first  the  student  had  to 
recognize  from  a  list  of  controls  the  correct  ones  to  set,  select  each 
control  in  turn,  and  perform  the  associated  equipment  action  on  the 
equipment  display.  As  each  control's  node  entered  the  recall  state, 
each  control  still  had  to  be  selected,  but  its  name  had  to  be  recalled 
from  memory.  In  each  of  these  states,  the  associated  equipment  action 
was  taken  only  after  the  control  was  selected.  Recording  of  student 
performance  continued  and  after  the  criterion  number  of  correct 
performances  was  reached,  the  state  of  the  "Set  Controls"  node  changed 
to  performance.  The  next  time  the  task  "Set  Controls  "  would  have  been 
encountered  with  its  associated  task  display,  no  task  display  was 
presented;  instead,  all  equipment  displays  under  this  node  were  run  as  a 
single,  integrated  equipment  display  without  intervening  task  displays. 
Finally,  when  the  set  controls  node  reached  the  summary  state,  on 
selecting  "Set  Controls"  the  trainee  received  the  message,  "The 
voltmeter  controls  have  been  set  pioperly  for  measuring  the  desired 
signal . " 
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VOLTMETER 


Training  Sequence  Before  Training  Sequence  After 

1.  Enter"Sef  up  voltmeter"  1.  Enter" Set  up  voltmeter” 

2.  Enter"Set  range  control"  2.  Set  range  to  10 

3.  Set  range  to  10  3.  Set  function  to  KOHMS 

4.  Enter  Set  function  control"  4.  Set  print  to  TRACK 

5.  Set  function  to  KOHMS 

6.  Enter" Set  print  control" 

7.  Set  print  to  TRACK 


As  a  consequence  of  demonstrated  proficiency  in  setting  voltmeter 
controls  (left-hand  training  sequence),  the  system  automatically  adapts  and 
subsequently  permits  the  separate  equipment  manipulations  to  be  chained 
together  as  one  integrated  equipment  action  (right-hand  training  sequence). 


Figure  4.  Chaining  equipment  manipulations. 


To  summarize,  the  task  of  setting  the  voltmeter  controls  was 
originally  four  separate  task  decisions,  each  followed  by  an  equipment 
action.  As  soon  as  mastery  of  the  separate  component  processes  was 
demonstrated,  the  task  of  setting  the  controls  became  one  single  task 
decision  followed  by  one  single  equipment  action.  Finally,  no  action  at 
all  was  required  since  the  trainee  had  already  demonstrated  complete 
competence.  Figure  5  provides  a  picture  of  the  nature  of  this  adaptive 
approach.  The  overall  effect  of  the  student  model  was  to  adapt  the 
level  of  required  interaction  according  to  trainee  achievement,  thus 
focusing  trainee  attention  on  unmastered  portions  of  the  task.  This 
provided  the  potential  for  enhanced  training  effectiveness  and  reduced 
training  time  for  completing  a  set  of  simulations. 


Development  of  the  IGS  System 

The  IGS  system  developed  in  response  to  the  design  goals,  consisted 
of  two  major  equipment  components,  several  software  tools,  and  the 
initial  task  data  base.  The  overall  development  philosophy  was  to 
develop  the  system  on  a  large-scale  computer  and  then,  after  determining 
exactly  what  equipment  and  tools  were  required,  specify  the 
characteristics  of  a  stand-alone  system.  Conceptually,  the  system  was 
organized  into  the  problem-solving  component  and  the 
equipment-manipulation  component  (i.e.,  the  terminal  and  video  monitor, 
respectively).  The  problem-solving  component  consisted  of  those  system 
elements  devoted  to  building,  maintaining,  and  running  all  the  items 
associated  with  the  on-line  data  base  except  the  simulated  equipment 
manipulation  actions.  The  equipment  manipulation  component  consisted  of 
only  those  items  required  for  simulating  the  equipment  actions. 

The  IGS  Problem-Solving  Component 

The  computer  used  for  this  component  was  a  Control  Data  Corporation 
(CDC)  Cyber  system  located  at  the  Air  Force  Human  Resources  Laboratory 
(AFHRL),  Training  Systems  Division,  on  Lowry  AFB,  Colorado.  Some  of  the 
software  tools  already  available  on  that  system  were  used  simply  to 
avoid  unnecessary  duplication  of  development  effort.  The  tools  used 
included  the  existing  text  editor  and  the  existing  graphics  editor.  The 
tools  developed  specifically  for  the  IGS  system  included  a  data  base  and 
simulation  editor,  a  simulation  run-time  program,  a  student  enrollment 
and  data  analysis  program,  and  several  print  and  report  utilities.  All 
of  these  programs  were  written  in  the  CAMIL  programming  language. 

The  Global  Task  Hierarchy  (GTH)  editor  program  performed  the 
functions  of  entering,  deleting,  renaming,  and  classifying  tasks  in  the 
hierarchical  data  base.  In  addition,  the  program  permitted  the  SMEs  to 
specify  which  tasks  were  relevant  to  a  particular  simulation  (i.e., 
problem  protocol)  and  the  order  in  which  the  tasks  should  be 
accomplished  for  that  protocol.  Finally,  this  editor  was  used  to  build 
the  task  displays  with  feedbacks  and  equipment  displays.  In  essence, 
the  GTH  editor  was  the  heart  of  the  system. 


The  simulation  run-time  program  used  the  task  data  base,  the 
specific  protocol  task  orderings,  the  task  and  equipment  displays,  and 
the  student  model  to  present  the  simulations  to  the  trainees.  The  run- 


Prior  to  demonstration  of  competence  on  a  task  component,  the 
adaptive  model  requires  the  trainee  to  verbalize  explicitly  task  planning  and  deci¬ 
sions  before  making  the  associated  equipment  manipulations.  After  competence 
has  been  demonstrated,  the  planning  and  decision  structure  has  been  internalized 
by  the  trainee,  who  may  now  interact  with  the  equipment  directly. 


Figure  5.  Adaptive  simulation. 


time  program  also  updated  the  student  model  information  and  was  the  link 
to  the  videodisc-based  equipment  component. 


The  student  enrollment  program  was  used  to  specify  courses  and  the 
set  of  problem  protocols  that  made  up  each  course.  It  was  also  used 
to  specify  whether  or  not  the  protocols  were  ordered  and  whether  or  not 
a  protocol  was  required  or  optional.  Finally,  this  program  was  used  to 
enroll  students,  assign  protocols  to  the  student  for  each  simulation 
session,  and  maintain  total  time  and  score  records  by  student  for  each 
protocol  worked. 

Print  utilities  were  developed  for  the  task  data  base  as  well  as 
the  task  choices  for  each  protocol.  In  addition,  a  screen  dump  utility 
was  developed  for  the  detailed  student  performance  records.  Finally,  a 
screen  print  routine  was  developed  for  the  graphics  used  in  the  task  and 
equipment  displays. 

The  Equipment -Manipulation  Component 

This  component  of  the  IGS  was  assembled  entirely  of  off-the-shelf 
commercial  hardware.  The  computer  used  for  the  equipment-manipulation 
component  was  an  Apple  II  microcomputer.  This  computer  was  augmented 
with  five  special  cards  to  interface  to  other  peripheral  devices.  The 
devices  included  a  laser  videodisc,  a  15-inch  color  video  monitor,'  a 
graphics  overlay  generator,  a  touch  pad,  and  a  medium-resolution  color 
terminal.  The  software  for  this  component  was  entirely  originally  built 
software.  The  primary  programs  included  a  system  picture  editor,  a  step 
file  builder,  a  video  graphics/text  editor,  two  run-time  programs,  and 
several  picture  file  utilities.  All  programming  for  the  equipment 
component  was  done  in  assembly  language  and  Pascal. 

The  system  picture  editor  was  used  to  develop  a  data  base 
describing  the  contents  of  the  videodisc.  This  data  base  was  a 
structured  data  base  reflecting  the  relationship  of  the  separate 
pictures  on  the  disc  to  one  another.  Each  entry  contained  the  name  and 
the  frame  number  of  the  picture,  and  the  names  of  the  pictures  that  were 
closeups  of  the  elements  of  this  picture.  The  program  was  used  to 
create,  delete,  and  name  picture  records  and  to  identify  touch  pad 
coordinates  of  picture  elements. 

The  step  file  builder  was  used  to  create  files  describing  the 
sequence  of  touch  pad  coordinates  and  picture  usage  for  a  particular 
step  associated  with  an  equipment  node  used  in  a  protocol.  This  program 
was  not  an  editor,  but  simply  a  file  creation  program.  The  program  took 
as  input  the  system  picture  file  and  author  touch  sequences  to  build  the 
desired  step  file.  The  program  could  specify  both  picture  and  motion 
sequences  as  elements  of  a  step.  In  addition,  a  digitized  graphic 
overlay  could  be  specified  by  file  name  for  use  with  a  particular  disc 
image . 

The  video  graphic  text  editor  was  used  to  create  and  modify 
digitized  graphic  overlay  files  that  could  be  superimposed  on  a  video 
image  using  the  graphic  overlay  generator.  The  generated  files 
contained  text  and  simple  graphics  and  were  stored  on  the  microcomputer 
floppy  disks. 
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The  two  run-time  programs  were  the  author  run-time  and  IGS  run¬ 
time.  The  author  run-time  program  was  used  to  check  each  of  the  step 
files  for  accuracy  and  completeness  before  using  the  step  with  trainees. 

This  program  took  all  input  from  the  terminal  keyboard  and  touch  pad  and  • 

ran  the  step  without  the  associated  problem-solving  terminal  displays. 

The  IGS  run-time  program  performed  exactly  as  the  author  run-time  except 

action  was  controlled  by  the  problem-solving  component  in  place  of  the 

keyboard.  Both  programs  scored  the  student's  performance  on  each  step, 

noting  whether  the  correct  touch  areas  were  used  in  the  required 

sequence.  The  author  program  displayed  the  results  on  the  terminal  • 

screen  whereas  the  IGS  run-time  program  transmitted  the  result  to  the 

Cyber  computer  system  for  use  in  updating  the  student  model. 

The  file  utilities  included  print  programs  for  both  the  system 
picture  file  and  the  step  files.  In  addition,  a  system  picture  check 

program  and  a  f ix  program  were  developed  for  checking  and  maintaining  • 

the  integrity  of  the  system  picture  file.  The  remainder  of  the 

available  file-handling  utilities  were  those  supplied  with  the  Pascal 
operating  system. 


Development  of  the  Task  Data  Base 

The  approach  taken  in  the  development  of  on-line  task  analysis  was 
to  build  a  skeleton  task  tree  through  traditional  front-end  analysis, 
employing  the  technique  of  hierarchical  problem  decomposition.  The 
initial  data  base  was  then  refined  and  filled  out  as  a  consequence  of 
using  it  to  build  simulations. 

After  entering  the  .skeleton  tree  into  the  task  data  base  using  the 
authoring  editor,  an  attempt  was  made  to  create  a  simulation  using  what 
was  available  on-line  to  that  point.  The  top  node  provided  the  first 
set  of  decision  alternatives,  which  were  adequate  for  assigning  the 
first  correct  choices.  As  each  correct  choice  was  selected  to  be  the 
new  current  node,  new  sets  of  decision  alternatives  were  available  and 
assigned  as  correct  choices.  Eventually  a  current  node  was  reached  for 
which  one  or  more  decision  alternatives  (i.e.,  children)  were  missing. 
At  this  point,  the  authoring  editor  was  used  to  add  children  to  the  list 
of  alternatives  for  the  current  node. 


For  example,  suppose  the  only  existing  decision  alternatives  for 
the  current  node  were  "Complete  Block  Diagram"  and  "Check  Signals." 
Further,  suppose  the  protocol  being  developed  required  that  a  visual 
inspection  be  performed  after  completing  the  block  diagram  and  before 
checking  signals.  Using  the  authoring  editor,  the  new  child  "Perform 
Visual  Inspection"  was  added  to  the  list  of  decision  alternatives  for 
the  current  node.  Once  "Perform  Visual  Inspection"  was  available  as  a 
decision  alternative,  it  was  assigned  as  the  second  correct  choice. 

When  "Perform  Visual  Inspection"  was  selected  as  the  current  node, 
no  decision  alternatives  were  available  because  it  was  a  new  node.  At 
this  point,  new  decision  alternatives  were  added  as  children  of  "Perform 
Visual  Inspection".  Once  the  alternatives  were  added,  they  were 
assigned  as  correct  choices,  and  the  process  was  repeated.  Through  this 


iterative  process,  the  task  data  base  was  filled  out  and  refined  as  a 
direct  consequence  of  developing  new  problem  protocols. 


Since  the  menus  that  appeared  at  the  bottom  of  each  task  display 
were  read  at  simulation  run  time  from  the  task  data  base,  all  changes  to 
the  data  base  affected  even  simulations  previously  developed.  Each  new 
decision  alternative  appeared  automatically  on  the  appropriate  task 
display. 

This  approach  to  the  development  of  the  task  data  base  for 
simulation  had  the  major  advantage  of  ensuring  that  the  task  data  were 
always  relevant  to  the  simulations  developed.  The  strategy  was  to  let 
the  demands  of  the  simulations  cause  the  on-line  task  analysis  to  grow. 
The  task  analysis  represented  by  the  task  data  base  and  the  tasks 
emulated  by  the  trainees  were  exhaustive  and  representative  of  the  job 
to  the  same  extent  that  the  set  of  problem  protocols  used  to  develop  the 
simulations  was  complete  and  representative  of  the  job. 

Summary  of  IGS  System  Design 

The  design  and  development  of  the  IGS  system  was  an  attempt  to 
successfully  produce  a  training  effectiveness  advantage  over  traditional 
instructional  approaches  involving  actual  equipment  and  equipment 
simulators.  The  main  feature  of  the  system  was  focusing  the  simulations 
on  the  emulation  of  job  tasks  as  opposed  to  equipment  operation.  A 
second  feature  of  the  IGS  system  was  adapting  the  simulation 
presentations  to  student  achievement.  To  do  this,  the  structure  of  the 
simulations  was  altered  in  a  way  that  focused  trainee  attention  on 
unmastered  tasks,  eliminating  unnecessary  repetitive  practice  of  learned 
skills. 

The  IGS  system  also  provided  instructional  development  assistance 
to  the  SMEs.  Simulations  were  simply  created  by  flagging  a  path  through 
an  on-line  task  data  base  representing  the  set  of  goals  and  actions 
employed  in  accomplishing  the  job  tasks.  This  approach  was  robust 
enough  to  support  the  development  of  part-task  simulations  and 
alternative  troubleshooting  approaches. 


III.  IGS  SYSTEM  IMPLEMENTATION 


Training  Analysis 

The  purpose  of  the  training  analysis  was  to  determine  the  most 
suitable  role  of  the  graphics  simulator,  considering  the  nature  of  the 
course  in  which  it  would  be  used  and  the  instructional  potential  of  the 
graphics  simulator.  The  present  section  describes  the  rationale  and 
impetus  for  many  of  the  IGS  design  features  presented  in  the  previous 
chapter.  It  also  presents  the  rationale  for  what  was  considered  the 
optimal  role  for  the  IGS  within  the  existing  course  structure. 

The  target  block  of  instruction  was  Block  IX  of  the  F-lll 
Integrated  Avionics  Computerized  Test  Station  and  Component  Specialist 
course  (3ABR32634A)  at  Lowry  AFB.  This  block  taught  the  maintenance  and 
operation  of  the  6883  Test  Station,  an  intermediate  level  automatic  test 
equipment  stand. 

The  training  analysis  was  performed  using  the  standard  Air  Force 
course  documents  as  a  starting  point.  These  included  the  course  Plan  of 
Instruction  (POI),  the  Specialty  Training  Standard  (STS),  and  the  Lesson 
Plans.  An  outline  of  instructional  topics  was  constructed  identifying 
the  training  tasks  addressed  in  the  course  documents.  In  the  course  of 
instruction,  all  of  the  test  station  and  most  of  its  associated  Line 
Replaceable  Units  (LRUs)  were  covered.  The  two  major  areas  of 
instruction  were  related  to  the  two  major  types  of  LRU  signals  (i.e., 
analog  and  digital). 

The  organization  of  the  curriculum  featured  presentation  of  theory 
followed  by  hands-on  practice  at  the  end  of  the  block.  Theory, 
function,  and  signal  flow  of  Test  station  Replaceable  Units  (TRUs)  were 
presented  first  for  the  analog  units  and  then  for  the  digital  units. 
Hands-on  practice  in  running  the  maintenance  tapes  for  the  test  station 
(TS),  the  Feel  and  Trim  (F&T)  LRU,  and  the  Multiplexer  Converter  Set 
(M/S)  LRU  was  provided  near  the  end  of  the  block.  The  F&T  was  largely 
analog;  the  M/S,  largely  digital.  See  Table  1  for  an  overview  of  the 
time  spent  in  the  various  Block  IX  units  as  listed  in  the  course  POI. 
Units  1  through  9  are  the  units  covering  tie  TRUs,  while  10  through  13 
contain  practical  exercises  for  the  test  station,  as  well  as  theory 
and  practical  exercises  for  the  LRUs.  The  total  course  length  was  72 
hours  or  9  days,  with  a  total  of  16  hours  of  practical  excercises. 

A  series  of  interviews  was  held  with  the  course  instructors  to 
identify  the  instructional  process  and  ways  the  test  station  was  used 
instruct iona 1 ly .  The  instructors  also  provided  protocols  for  test 
station  operation,  including  descriptions  of  the  rationale  for  various 
steps . 


The  process  of  detailing  such  an  analysis  is  always  personnel 
intensive  and  time-consuming.  The  first  tasks  analyzed  revealed  that 
several  of  the  equipment  and  decision  steps  were  repeated  across 
problems.  For  example,  the  oscilloscope  was  repeatedly  set  up  to  make 
time  delay  measurements  and  the  digital  voltmeter  was  repeatedly  set  up 
to  make  voltage  measurements.  The  planning  steps  leading  to  the 


Table  1.  Block  IX  Units  and  Time  Allocation 


Unit 

Title 

Unit 

Title 

1. 

Test  Station  (TS)  Introduction 
T  -  1  hour 

8. 

Signal  Converter  Unit 

T  -  1 . 5  hours 

2. 

Switching  Complex 

T  -  6  hours 

9. 

Digital  Interface  Unit 

T  -  1.5  hours 

3. 

Switching  Control  Unit 

T  -  4  hours 

10. 

Test  Station 

P  -  12  hours  * 

4. 

Flight  Control  System  Adapter 

T  -  4  hours 

11 . 

Terminology 

T  -  2  hours 

5. 

Signal  Converter  Simulator 

T  -  8  hours 

12. 

Feel  and  Trim 

T  -  8  hours  P  -  2  hours  * 

6. 

Serial  Digital  Adapter 

T  -  4  hours 

13. 

Multiplexer/Converter  Set 

T  -  9.5  hours  P  -  2  hours  * 

7. 

Parallel  Digital  Adapter 

Misc  Training  and  Testing 

T  -  1.5  hours  5  hours 


T  =  Theory  and  Function  P  =  Practical 
*  =  AET  and  IGS  training,  totaling  16  hours 

selection  of  these  devices  and  the  measurement  of  these  signals  were 
also  analyzed  in  order  to  integrate  commonly  recurring  steps  in  separate 
problems  into  a  global,  overall  representation  of  the  job.  One  outcome 
was  the  identification  of  the  overall  job  logic  (See  Figure  6).  The 
flow  depicted  has  implicit  decision  points  which  became  explicitly 
represented  in  the  various  simulations. 


Troubleshoot  LRU 
Using  Auto-Test 


No  Test  Failure  Test  Fails  at  Test  Number 

3)  4) 

Technical  Order  (T.O.)  T.O.  Corrective  Action 
Corrective  Action  Doesn't  Repair  Problem 

Repairs  Problem  Same  Test  Number  Fails 


Manually  Troubleshoot  Manually  Troubleshoot 

Test  Station  LRU 


Figure  6.  Job  Logic. 


A  major  organizing  principle  for  the  development  of  the 
troubleshooting  simulations  was  discovered  during  the  in-depth  analysis 
of  the  equipment  and  instructor  protocols.  This  was  the  concept  of  a 
test  loop.  A  test  loop  consisted  of  the  parts  of  the  test  station  and 
LRU  (if  any)  involved  in  a  specific  test.  It  covered  the  stimulus 
source,  the  stimulus  switching,  the  stimulus  switching  logic,  the  test 
station/LRU  interface,  the  LRU  components,  the  response  signal 
interface,  the  response  logic,  the  response  switching,  and  the 
measurement  device.  If  a  programmed  test  failed  and  the  indicated 
repair  action  did  not  fix  the  problem,  the  technician  usually  began  by 
determining  which  area  of  the  test  loop  was  involved.  The  test  loop 
guided  the  manual  troubleshooting  process.  Although  the  words  "test 
loop"  were  identifed  with  regard  to  the  6883  Test  Station,  the  test 
configuration  implied  above  is  common  to  many  automatic  test  sets  and  is 
useful  when  manually  isolating  malfunctions. 

In  order  to  reinforce  the  test  loop  concept,  the  capability  to 
color-code  task  displays  by  region  of  the  test  loop  was  added  as  a 
feature  of  the  IGS  system.  This  coloring  scheme  associated  a  unique 
color  with  each  region  on  the  test  loop.  The  colors  were  presented  as 
borders  or  frames  around  the  task  displays.  For  example,  when  a  trainee 
was  troubleshooting  the  stimulus  portion  of  the  test  loop,  task  displays 
were  bordered  with  a  red  frame.  Task  displays  shown  when 
troubleshooting  the  measurement  device  portion  of  the  test  loop  were 
framed  with  blue.  In  the  first  simulation,  the  color-coding  scheme  was 
explained  when  trainees  first  saw  the  representation  of  the  test  loop 
and  its  seven  regions. 

A  loose  coupling  of  theory  and  practice  was  seen  in  the  temporal 
ordering  of  units  in  the  POI  (See  Table  1).  As  indicated  above,  the 
job  logic  involved  two  distinct  types  of  activity:  one  consisting  mainly 
of  known  sequential  actions,  requiring  little  or  no  understanding  of  the 
test  station,  and  another  being  mainly  problem  solving,  requiring  great 
amounts  of  test  station  understanding.  As  the  block  was  taught,  the 
theory  portions  of  the  block  were  applicable  to  the  problem-solving 
activities  and  were  reinforced  in  workbook  exercises.  However,  no 
formal  practice  involving  the  test  station  in  these  problem-solving 
activities  was  provided.  Formal  hands-on  practice  was  largely  limited 
to  the  fixed  procedure  activities.  A  Burtek  Patch  Panel  Trainer  (PPT) 
offered  some  part-task  training  in  signal  tracing  in  the  switching 
components  of  the  6883  Test  Station.  Practice  in  the  activities 

requiring  knowledge  of  job  logic  was  randomly  available  on  the  actual 
equipment  (when  it  failed  on  its  own  accord),  but  could  not  be 
structured  or  organized  for  the  purposes  of  training. 

The  lack  of  formal  problem-solving  instruction  in  daily  classroom 
training  indicated  a  need  to  support  troubleshooting  practice  on  a  daily 
basis.  If  the  IGS  was  to  have  an  impact  on  training  effectiveness,  it 
had  to  do  more  than  simply  represent  the  equipment  actions.  It  also  had 
to  represent  the  cognitive  activity  needed  for  troubleshooting 
competence  on  the  job.  Improvement  in  the  course  was  judged  achievable 
if  the  identified  omissions  in  the  course  organization  could  be  filled 
with  simulator  lessons.  Operating  within  the  constraints  of  the 
identified  role  of  the  simulator  required  that  these  lessons  be 


available  within  the  course's  current  curriculum  and  daily  activity 
plans . 

The  theory  part  of  the  block  was  augmented  with  simulations  which 
provided  part-task  practice  with  the  TRUs  under  study.  This  practice 
was  provided  using  a  troubleshooting  problem  scenario  in  which  a  failure 
had  been  isolated  to  a  particular  TRU.  Hence,  practice  in  situations 
involving  the  need  for  theoretical/functional  understanding  was  provided 
at  the  time  theory  was  taught.  Formal  maintenance  tape  practice 
sessions,  at  the  end  of  the  block,  were  augmented  to  include  both  fixed 
procedures  and  problem-solving  activities. 

The  requirement  of  daily  work  on  the  IGS  precipitated  a  study  of 
classroom  time  allocation.  The  analysis  included  classroom  activities 
and  equipment  utilization.  The  instructors  were  asked  to  reply  to  an 
informal  survey  of  how  their  time  was  spent  each  day  in  an  average 
classroom  week.  The  instructors  indicated  that  the  principal  activities 
were  reviewing,  lecturing,  and  demonstrating.  They  reported  the 
approximate  proportion  of  time  devoted  to  each  of  their  activities  (see 
Table  2). 

Table  2.  Overall  Percentage  of  Classroom 
Time  Use  in  Block  IX 


Lecture  and  Review 

30% 

Practical 

20% 

Homework  Review 

18% 

Workbook 

15% 

Break 

12% 

Other 

5% 

From  the  percentages  in  Table  2,  the  necessary  time  for  IGS  use  could  be 
obtained  during  the  normal  8-hour  classroom  day.  Using  the  "Practical" 
time  (approximately  90  minutes)  and  some  of  the  "Homework  Review"  time 
(approximately  10  minutes),  a  total  of  approximately  100  minutes  would 
be  available  each  day.  Over  the  9-day  block,  this  would  amount  to 
approximately  900  minutes  or  15  hours,  the  same  amount  of  time  set  aside 
in  the  regular  course  for  hands-on  AET  experience.  Assuming  a  nominal 
20  minutes  per  simulation,  four  students  per  class  (the  average),  and 
two  IGS  systems  in  the  classroom,  each  student  could  take  two 
simulations  each  day  within  the  available  time.  Maintaining  the  overall 
15  hour  time  limit  for  practical  exercises  was  the  desired  goal; 
therefore,  adjustments  to  insure  that  a  slow  IGS  student  did  not  get 
more  time  on  the  simulations  would  be  made  by  reducing  the  number  of 
simulations  worked.  Conversely,  an  IGS  student  who  was  quick  would  be 
permitted  to  work  additional  simulations  in  the  remaining  available 
time.  Additional  information  on  time  usage  in  the  IGS  treatment  is 
presented  in  Section  IV,  Evaluation. 


Simulation  Development 


Based  on  the  training  analysis  presented  in  the  previous  section,  a 
set  of  simulations  was  developed  to  support  the  existing  POI,  and  to 
replace  the  hands-on  AET  experience  normally  given  at  the  end  of  block 
IX.  The  simulations  began  with  part-task  protocols  supporting  the 
training  of  theory  and  functions  emphasized  in  the  early  portion  of  the 
block.  The  simulations  ended  with  whole-task  protocols  supporting  the 
practical  training  emphasized  near  the  end  of  the  block. 

A  team  of  Air  Force  instructors  was  asked  to  cover  each  objective 
in  the  POI  by  selecting  a  set  of  faults  on  which  representative 
simulations  would  be  based.  For  each  fault,  the  instructors  were  asked 
to  write  out  a  troubleshooting  protocol.  These  protocols  were 
paper-and-pencil  "story  boards"  of  the  problem  scenarios  and  model 
diagnostic  procedures  to  be  followed  in  isolating  the  respective  faults. 
Initially,  contract  personnel  used  these  protocols  to  begin  building  the 
on-line  task  analysis  data  base,  and  from  it,  the  sequence  of  decision 
points  which  constituted  the  core  of  the  simulations. 

Just  exactly  what  the  correct  or  preferred  diagnostic  procedures 
were  for  a  given  protocol  was  often  the  subject  of  heated  debate  among 
the  members  of  the  Air  Force  team.  In  general,  differing  points  of  view 
held  by  the  SMEs  were  brought  into  a  consensus  through  extensive 
discussion,  argument,  and  counterargument.  Soon  after  contractor 
personnel  developed  the  initial  on-line  hierarchical  task  data  base  for 
6883  Test  Station  maintenance,  Air  Force  instructors  began  generating 
new  protocols  on-line  rather  than  with  pencil  and  paper.  The  process  of 
on-line  task  decomposition  and  data  base  development  was  previously 
described  in  Section  II. 

In  addition  to  the  sequence  of  decision  points  which  defined  each 
simulation;  text  displays,  feedback  messages,  and  associated  videodisc 
sequences  were  developed.  The  Air  Force  SMEs  were  the  only  sources  of 
information  for  most  of  this  material  and  were  responsible  for  quality 
assurance,  both  in  terms  of  the  logical  consistency  of  the  on-line  task 
analysis  and  in  terms  of  editorial  modifications  to  input  text.  Table  3 
summarizes,  by  day,  the  series  of  simulations  developed  in  support  of 
the  POI  objectives.  Appendix  A  contains  screen  prints  of  a 
representative  simulation. 

Development  Effort 

The  effort  needed  to  develop  each  new  simulation  consistently 
decreased.  The  main  reason  for  this  was  that  the  on-line  task 
analysis,  which  supplied  the  decision  points  for  all  simulations,  became 
more  and  more  complete  as  each  simulation  was  developed.  As  a  result, 
less  and  less  effort  was  needed  to  expand  the  on-line  task  analysis  data 
base.  Since  each  new  simulation  used  major  portions  of  the  hierarchy 
already  developed,  less  and  less  new  material  needed  to  be  added.  For 
the  early  simulations,  the  majority  of  the  simulation  development  effort 
was  expended  in  building  the  task  data  base.  This  required  much  more 
effort  than  did  the  creation  of  the  associated  presentation  material 
(i.e.,  text  displays,  feedback  messages,  and  videodisc  image  sequences). 


Table  3.  Simulations  for  IGS  Instruction 


Day  POI  Topics 


Simulations 


1 

Introduction  and 

PRTCL10 : 

Part-task,  Fault 

=  Switching  Relay  11/044 

Switching 

PRTCL1 2 : 

Part-task,  Fault 

=  Stimulus  Relay  10/1 

2 

Switching  Control 

PRTCL7 : 

Part-task,  Fault 

=  Switching  Relay  1/031 

Unit,  Flight 

PRTCL4 : 

Part-task,  Fault 

=  Stimulus  Relay  02/1 

Control  System 

PRTCL1 1 : 

Part-task,  Fault 

=  Switching  Control  Unit 

(FCS)  Adapter 

PRTCL1 5 : 

Part-task,  Fault 

=  FCS  Adapter 

3 

Signal  Converter 

PRTCL13 : 

Part-task,  Fault 

=  3-wire  Test  Point  Relay 

Simulator  (SCS) 

4 

Serial/Parallel 

PRTCL14: 

Part-task,  Fault 

=  SCS  A41  Card 

Digital  Adapter, 

PRTCL16 : 

Part-task,  Fault 

=  SCS  Decoder  or  Transfer 

Signal  Converter 

Board 

Unit  Controller, 

PRTCL17  : 

Part-task,  Fault 

=  Digital  Interface  Unit 

Digital  Interface 

Output  Card 

Unit (DIU) 

PRTCL18 : 

Part-task,  Fault 

=  Serial  Digital  Adapter 

5 

Maintenance  Tapes 

PRTCL19: 

Whole-task,  Fault 

=  Micrologic  P/S  A1  Card 

I  and  II 

PRTCL20 : 

Whole-task,  Fault 

=  DATAC 

PRTCL21 : 

Whole-task,  Fault 

=  Test  Point/Stim  Relay 

PRTCL22 : 

Whole-task,  Fault 

=  A2/A9  12vdc  Breaker 

PRTCL23 : 

Whole-task,  Fault 

=  Digital  Drawer 

PRTCL24 : 

Whole-task,  Fault 

=  A2A3A21 

6 

Feel  and  Trim 

PRTCL1 : 

Part-task,  Fault 

=  Test  Point  Controller  A37 

PRTCL2 : 

Part-task,  Fault 

=  Phase  Sensitive  Converter 

PRTCL5 : 

Part-task,  Fault 

=  Stimulus  Relay  03/8 

PRTCL8 : 

Part-task,  Fault 

=  Signal  Generator  A28 

7 

Feel  and  Trim 

PRTCL6 : 

Whole-task,  Fault 

=  Center  Relay  Package  K48 

Maintenance  Tapes 

PRTCL25 : 

Whole-task,  Fault 

=  None 

PRTCL9 : 

Whole-task,  Fault 

=  Signal  Generator  TB3 

PRTCL26 : 

Part-task,  Fault 

=  Lower  Relay  Panel 

8 

Converter  Set 

PRTCL27 : 

Whole-task,  Fault 

=  None 

Maintenance  Tape 

PRTCL28 : 

Whole-task,  Fault 

=  Converter  Set 

9 

Converter  Set 

PRTCL3 : 

Whole-task,  Fault 

=  Converter  Set 

PRTCL29 : 

Whole-task,  Fault 

=  Converter  Set 

The  effort  required  to  develop  the  presentation  material  was  basically 
constant  since  it  needed  to  be  newly  developed  for  each  step  in  each 
simulation.  Overall,  the  development  effort  was  split  approximately  50- 
50  between  task  data  base  and  presentation  material  development. 

To  summarize,  the  effort  required  to  build  a  series  of  simulations 
had  two  basic  parts.  The  effort  associated  with  building  the  on-line 
task  data  base  continually  decreased  as  more  and  more  simulations  were 
developed.  The  effort  associated  with  flagging  the  correct  path  through 
the  task  hierarchy  and  annotating  the  path  with  associated  text 
displays,  feedback  messages,  and  videodisc  display  sequences  remained 
almost  constant. 

Videodisc  Data  Base  Development 

The  equipment  display  used  a  combination  of  videodisc  images  and 
graphic  overlays  to  represent  actual  equipment.  The  videodisc  medium 
was  chosen  for  its  large,  random  access  storage  capacity  at  reasonable 
cost.  The  graphic  overlays  were  added  to  represent  specific  equipment 
states  as  required  by  individual  lessons;  for  example,  to  change  the 
value  of  a  digital  voltmeter  display.  The  videodisc  image  base  was 
composed  of  over  20,000  still  frames  and  appropriate  motion  sequences 
used  to  represent  various  test  station  states  and  to  demonstrate 
the  attachment  of  LRUs,  adapters,  and  cables. 

In  addition  to  the  equipment  images  accessible  from  the  videodisc, 
several  internal  equipment  processes,  which  the  instructors  "picture" 
when  working  on  a  problem,  were  needed  to  complete  the  image  data  base. 
Many  of  these  added  images  were  dynamic  in  nature  and  were  better  suited 
to  representation  on  the  interactive  graphics  terminal  (e.g.,  wave 
shapes)  rather  than  the  videodisc.  These  images  covered  both  process 
visualizations  and  dynamic  equipment  interactions  and,  therefore,  were 
constructed  on  the  graphics  terminal  using  a  graphics  editor. 

The  images  included  in  the  videodisc  image  data  base  were  selected 
by  experienced  Air  Force  Training  Command  (ATC)  instructors  considering 
the  proposed  IGS  content.  A  filming  schedule  of  test  station  and  LRU 
component  images  was  developed.  This  schedule  included  images  of  the 
overall  test  station,  of  the  four  test  station  bays  or  racks,  and  of 
individual  TRUs.  Switches,  dials,  meters,  display  windows, 
representative  fuse  holders,  and  test  points  on  the  TRUs  were  also 
included.  In  addition,  LRU  adapters,  cables,  and  "black  boxes"  were 
scheduled  to  be  photographed.  Finally,  components  within  TRUs  and  LRUs 
(replaceable  boards  as  well  as  components  attached  to  the  boards)  were 
included  as  necessary. 

A  16-mm  movie  camera,  set  on  single-frame  exposure  except  for  a  few 
motion  sequences,  was  used  to  photograph  the  test  station  according  to 
the  filming  schedule.  The  6883  Test  Station  located  on  Lowry  AFB  was 
used  for  all  photography.  Multiple  exposures  of  each  shot  were  taken  to 
bracket  calculated  exposures,  A  photographer,  schedule  coordinator,  and 
SME  (i.e.,  an  ATC  instructor)  accomplished  all  photography. 


The  resulting  16-mm  film  was  processed  and  run  on  film  editing 
machines  to  inventory  each  frame.  A  picture  catalog  of  numbered  frames 
was  prepared  for  use  during  development  of  the  simulations.  The  film 
was  then  transferred  to  video  tape  and  finally,  to  videodisc. 

A  videodisc  image  data  base  was  built  using  the  picture  catalog 
with  frame  numbers.  Simulated  equipment  manipulation  sequences  were 
built  from  the  data  base  for  use  by  the  students  at  various  points  in 
the  lessons  (i.e.,  problem  protocols).  Graphic  overlays  were  added  as 
needed.  These  overlays  allowed  lesson  developers  to  add  meter  readings, 
switch  settings,  light  bulb  and  fuse  holder  illumination,  and  other 
visual  cues  onto  videodisc  images  as  appropriate  for  specific  lessons. 


Simulation  Adequacy 


Several  attributes  of  the  simulations  and  their  development 
provided  confidence  in  the  adequacy  of  the  simulations.  First,  the 
simulations  were  developed  by  Air  Force  SMEs  and  were  closely  tied  to  an 
approved  course  POI.  In  addition,  the  SMEs  reached  consensus  concerning 
the  appropriate  diagnostic  approach  employed  in  each  simulation. 


There  are  two  basic  ways  to  establish  the  acceptability  of 
instructional  material.  By  far  the  most  common  is  to  have  the  material 
reviewed  and  critiqued  by  a  panel  of  SMEs  and  to  arrive  at  a  group 
consensus  about  the  form  and  content  of  the  material.  The  second  way  to 
establish  acceptability,  in  accordance  with  ISD  procedures,  is  to: 


1.  articulate,  in  advance,  the  objectives  for  the  material 

2.  develop  an  independent  measure  of  objective  mastery 

3.  use  the  material  to  train  naive  subjects 

4.  test  after  training  for  objective  mastery 


In  the  case  of  the  IGS  materials,  SME  group  consensus  was  considered 
adequate  for  determining  materials  acceptability. 
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IV.  EVALUATION 


Overv iew 

Evaluation  of  the  IGS  system  covered  three  major  areas:  Training 
Effectiveness,  Environmental  Suitability,  and  Cost.  The  first  part  of 
this  section  describes  the  evaluation  components  and  the  second  section 
presents  the  evaluation  results. 

Training  block  IX  of  the  Integrated  Avionics  Computerized  Test 
Station  and  Component  Specialist  (3ABR32634A)  course  of  instruction  was 
assigned  as  the  target  of  this  study.  This  9-day  block  covered 
operation  and  maintenance  skills  for  the  6883  Test  Station.  The 
training  emphasized  development  of  diagnostic  and  procedure-following 
competence.  This  block  of  instruction  was  administered  to  two  treatment 
groups,  inst ruct iona 1  ly  similar  except  for  the  training  device  used. 

The  control  group  used  an  actual  equipment  trainer  (AET)  as 
prescribed  by  the  course  POI.  During  the  last  2  days  of  the  9  days  of 
instruction,  the  AET  was  used  to  provide  approximately  15-16  hours  of 
hands-on  experience  in  performing  prescribed  procedures  and  diagnostic 
testing . 

The  experimental  group  used  the  IGS  in  place  of  the  AET.  The  IGS 
was  used  throughout  the  9-day  block,  to  provide  practice  in  following 
prescribed  procedures  and  solving  troubleshooting  problems.  Instructors 
limited  IGS  contact  time  to  approximately  100  minutes  per  day,  as 
described  in  Section  III.  Any  adjustments,  required  to  stay  within  the 
time  limit,  were  made  by  reducing  the  number  of  simulations  worked. 
Therefore,  both  IGS  and  AET  students  received  a  total  of  approximately 
16  hours  of  training  on  their  respective  training  devices. 

Two  tests,  unique  to  this  study,  were  specially  developed  to 
measure  the  capabilities  of  the  two  treatment  groups:  an  actual 
equipment  hands-on  test  of  precedure-fol lowing  capability,  and  a 
paper-and-pencil  test  of  troubleshooting  reasoning  competence  for 
equipment-specific  fault  isolation  problems.  A  projected  job 
proficiency  test  from  a  previous  study  and  the  normal  end-of-block  test 
were  also  used  to  measure  each  student's  competence.  Finally,  student 
opinions  were  surveyed.  AET  and  IGS  equipment  costs  were  compared  and 
IGS  suitability  to  the  training  environment  was  evaluated. 

Evaluation  Component s 

This  section  presents  descriptions  of  the  two  experimental 
treatments,  of  the  evaluation  test  instruments,  of  the  subject 
assignments  to  treatment  groups,  of  the  testing  schedule,  and  of  the 
data  collection. 

Experimental  Treatment s 

The  AET  control  treatment  included,  classroom  training  using  the 
standard  instructor  POI  and  AET  (i.e.,  the  6883  Test  Station)  hands-on 
training  during  the  last  two  days  of  instruction  block  IX. 


The  IGS  experimental  treatment  included  classroom  training  using 
the  standard  instructor  POI  and  hands-on  training  using  the  IGS  system 
(i.e.,  both  the  problem  solving  and  equipment  manipulation  components). 
The  primary  difference,  other  than  the  nature  of  the  practice 
simulations  on  the  IGS,  was  the  IGS  daily  distributed  practice  (100 
minutes  for  9  days)  rather  than  the  AET  massed  practice  (8  hours  for  the 
lasc  2  days) . 

The  number  of  trainees  available  for  the  study  in  the  avionics 
maintenance  career  path  (AFSC  3ABR32634A)  was  43.  Because  of  the  low 
number  of  trainees,  the  experimental  treatments  were  limited  to:  AET 
training  with  no  IGS  system  contact  and  IGS  training  with  no  AET  system 
contact . 

Training  Modes 

AET  training .  AET  training  for  the  3ABR32634A  course  instructional 
block  (Block  IX)  consisted  of  7  days  of  classroom  instruction  and  2  days 
of  practical  instruction  on  the  6883  Test  Station  (Table  4).  The  7  days 
of  classroom  instruction  consisted  of  stand-up  lecture,  chalkboard 
diagramming,  and  workbook  problem  assignments.  During  this  instruction, 
the  students  were  taught  about  the  test  station  Replaceable  Units  (TRUs) 
and  the  aircraft  Line  Replaceable  Units  (LRUs)  that  the  test  station 
checked.  The  lecture  and  chalkboard  instruction  focused  primarily  on 
the  electronic  circuitry,  programming,  and  component  identification  of 
the  TRUs  and  LRUs.  The  workbook  problems  stressed  troubleshooting 
procedures  and  processes.  Samples  of  the  workbook  troubleshooting 
problems  are  included  in  Appendix  B. 

Table  4.  Instructional  Approach  by  Training  Mode 


Objective 


_ Training  Mode _ 

Actual  Equipment  IGS 


Instruction  in 
equipment  familiar' 


Stand-up  lecture 
Chalktalk 

Workbook  assignments 


Stand-up  lecture 
Chalktalk 
Workbook 
assignments 


Instruction  in 

troubleshooting 

techniques 


CO 

CO 


r*-. 


Workbook  assignments 


CO 

>> 

CO 

T3 

ON 


Workbook 
assignments 
IGS  simulations 


w 

to 

-u 


Instruction  in 
equipment  operation 


Hands-on  opei  ation 
of  6883  test  station 


IGS  simulations 


The  practical  portion  of  the  AET  training  consisted  of  2  days  of 
equipment  operation.  Due  to  restrictions  on  inducing  failures  in  the 
6883  Test  Station  and  to  the  fact  that  operational  equipment  was  not 
designed  to  function  as  training  equipment,  the  role  of  the  AET  during 
the  practical  instruction  was  primarily  one  of  performing  equipment 
operational  checks  without  fai1  res.  Although  the  actual  equipment  is 
operated  by  a  single  technician  in  the  field,  the  limited  number  of  test 
stations  in  the  classroom  (only  one),  together  with  safety  and  cost 
considerations,  necessitated  a  group  approach  to  the  practical 
exercises.  This  approach  was  used  by  the  AET  group  during  the  entire 
evaluation  period. 

The  AET  7-day  c lassroom/ 2-day  practical  training  approach  was 
selected  by  ATC  after  considering  other  alternatives.  At  one  point, 
classroom  training  was  conducted  in  the  test  station  equipment 
laboratory  in  order  to  permit  distributed  practice  on  the  AET  over  the 
entire  Block  IX  training  period.  This  distributed-practice  approach  was 
subsequently  rejected  in  favor  of  the  current  7-day/ 2-day 
massed-practice  approach.  In  addition,  the  massed-practice  approach  was 
used  in  all  AET  related  blocks  of  instruction  prior  to  Block  IX.  The 
AET  massed-practice  approach  \  s  considered  by  ATC  to  be  the  most 
practical  and  was  the  approved  approach. 

IGS  training .  A  similar  standardized  IGS  format  was  implemented 
for  Block  IX  training.  IGS  training  was  identical  to  AET  training  in 
several  ways.  The  same  POI  served  as  the  guide  to  sequencing  the 
training  topics  and  activities.  Also,  the  IGS  students  had  the  same 
overall  time  in  instruction  (9  days),  used  the  same  study  guides  and 
technical  orders,  received  the  same  chalktalk  for  programming  and 
diagramming  examples,  used  the  same  workbook  problems  for 
troubleshooting  procedures  and  processes,  and  had  the  same  amount  of 
time  per  class  for  practical  test  station  instruction  (16  hours  or  2 
days ) . 

The  principal  difference  between  IGS  and  AET  training,  besides  the 
use  of  the  IGS  to  replace  the  AET,  was  in  the  content  of  the  practice 
problems  and  their  usage  pattern.  As  described  in  Section  III,  the 
simulations  were  part-  and  whole-task  job  samples  containing  both  the 
cognitive  elements  and  the  equipment  manipulation  aspects  of  6883  Test 
Station  operation  and  maintenance.  In  general,  the  problems  were 
troubleshooting  problems  typical  of  those  found  on  the  job.  Usage  of 
the  problems  was  distributed  throughout  the  9-day  training  period 
instead  of  being  massed  during  the  last  two  days.  This  approach  was 
selected  to  make  optimum  use  of  the  IGS  system  capabilities  and  to 
effectively  integrate  them  into  the  training  curriculum. 

Initially,  27  simulation  protocols  were  developed.  After  pilot 
testing  the  protocols,  it  was  determined  that  the  average  protocol  would 
take  approximately  45  minutes.  Assuming  two  students  per  IGS  class 
(i.e.,  one-half  of  the  usual  4  students  per  class),  each  student  could 
work  only  one  protocol  per  day  in  the  100  minutes  available.  This  would 
mean  that  a  total  of  only  nine  protocols  could  be  worked  during  the 
9-day  block.  Since  a  consistent  set  of  protocols  was  desired  for  all 
IGS  students,  11  of  the  27  were  chosen  for  use  during  the  evaluation 


period.  The  11  protocols  were  chosen  to  maximize  coverage  of  the  test 
station  components,  to  support  daily  teaching  objectives,  to  include 
both  part-  and  whole-task  protocols,  and  when  there  were  conflicts,  to 
support  instructor  preferences.  Once  chosen,  the  protocols  were 
sequenced  in  accordance  with  daily  teaching  objectives  and  all  students 
took  the  problems  in  the  same  order. 

Two  IGS  student  stations  were  installed  in  the  classroom  to  insure 
that  each  student  would  receive  45  minutes  of  simulation  practice  each 
day  even  if  there  were  four  students  in  an  IGS  class.  Students  operated 
the  IGS  stations  on  an  individual  basis.  No  contact  with  the  6883  Test 
Station  was  permitted  for  the  IGS  trainees.  While  two  students  each 
operated  a  separate  IGS  station,  the  remaining  students  (maximum  of  two 
in  any  class)  solved  workbook  problems  at  their  desks  until  it  was  their 
turn  to  use  the  stations.  Using  this  approach,  IGS-trained  students 
could  be  perceived  as  having  had  more  troubleshooting  training  than  AET 
students.  However,  no  IGS  students  had  any  actual  equipment  ope-ational 
procedures  training.  IGS  student  practical  training  consisted  of  6883 
simulated  equipment  manipulation  and  troubleshooting  exercises  using  the 
IGS . 

The  number  of  students  who  took  IGS  training  was  21.  Of  these, 
classroom  training  data  was  lost  on  three  students  due  to  a  computer 
crash  and  backup  failure.  Of  the  remaining  18  students,  one  completed 
only  the  first  three  protocols  (PRTCL10,  PRTCL11,  and  PRTCL14)  of  the 
eleven  available.  Four  students  completed  three  additional  protocols 
(PRTCL18,  PRTCL19 ,  and  PRTCL8) ,  for  a  total  of  six.  Approximately 
one-half  of  the  IGS  students  (10)  completed  a  seventh  protocol  (PRTCL6). 
One  student  completed  the  eigth  protocol  (PRTCL17)  and  two  students 
completed  all  protocols  through  the  ninth  protocol  (PRTCL1).  The 
maximum  number  of  protocols  worked  by  any  student  was  nine. 

As  discussed  in  the  intoductory  chapter,  this  project  adopted  the 
goal  of  using  simulators  to  emulate  not  only  the  equipment  but  also  the 
task  decisions  made  by  the  skilled  maintenance  technician.  Thus,  the 
optimal  role  of  the  IGS  system  was  seen  as  providing  both 
troubleshooting  and  simulated  equipment  manipulation  training 
distributed  over  the  9-day  period  of  the  block.  In  this  format,  the  IGS 
simulations  were  enhancements  to  the  daily  objectives  of  the  POI  (ref. 
Table  3). 

Test  Instruments 

In  the  training  effectiveness  analysis,  several  measurement 
instruments  were  used.  These  included  existing  measurement  instruments 
developed  and  administered  by  the  Air  Force  and  measures  developed  by 
the  Denver  Research  Institute  (DRI).  This  section  describes  all  the 
measurement  instruments,  including  the  rationale  behind  their 
development . 

Air  Force  test  instruments .  The  following  test  instruments  were 
developed  either  by  the  Armed  Services  or  by  the  Air  Force. 
Administration  of  these  measures  to  evaluate  subjects  was  the  sole 
responsibility  of  the  Armed  Services  or  the  Air  Force. 
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The  Armed  Services  Vocationa 1  Aptitude  Battery  (ASVAB) .  The  ASVAB 
is  a  DoD  multiple-choice  paper-and-pencil  test  instrument  developed 
jointly  by  all  the  Armed  Services.  The  ASVAB  is  composed  of  10  subtests 
that  measure  various  types  of  aptitudes.  The  services  combine  these 
subtests  in  different  ways  to  form  composite  scores  used  for  selecting 
and  classifying  enlistees.  The  Armed  Forces  Qual if ication  Test  (AFQT) 
composite  test  score  was  designed  to  assess  the  general  training 
potential  of  military  enlistees  and  is  primarily  used  for  initial 
service  selection  by  all  services. 

The  Air  Force  uses  the  following  ASVAB  composite  or  aptitude  index 
scores  for  job  selection  and  classification:  General,  Mechanical, 
Administrative,  and  Electronics.  Students  entering  the  3ABR32634A 
course  generally  require  a  minimum  electronics  score  of  80,  although 
waviers  are  permitted  for  cross-trainees  and  other  special  cases.  The 
ASVAB  is  administered  by  a  trained  test  administrator  from  one  of  the 
military  services  at  the  time  of  initial  enlistee  screening.  A  detailed 
description  and  development  of  ASVAB  Forms  8,9,  and  10  is  found  in  Ree, 
Mathews,  Mullins,  and  Massey  ( 1982)  and  in  Ree,  Mullins,  Mathews,  and 
Massey  (1 982). 

Instruct iona 1  B lock  Test .  This  end-of-block  test  is  designed  to 
measure  how  well  students  have  learned  the  material  presented  in  the 
classroom  during  specific  instructional  blocks.  It  is  a  pencil- 
^nd-paper,  multiple-choice  test  developed  by  ATC  instructor  personnel. 
This  test  is  normally  administered  by  the  classroom  instructor  on  the 
last  day  of  each  instructional  block. 

IGS  evaluation  test  instruments .  Test  instruments  were  designed  and 
developed  by  DRI  to  measure  performance  of  Air  Force  subjects  involved 
in  this  study.  Although  DRI  personnel  were  involved  in  the  design, 
development,  and  evaluation  of  the  IGS  system,  those  individuals 
involved  in  developing  the  measurement  instruments  and  conducting  the 
evaluation  were  not  involved  in  the  design  or  development  of  the  IGS 
system  or  its  simulations.  The  DRI  test  instruments  were  administered 
to  all  evaluation  subjects  on  the  day  after  the  Block  IX  end-of-block 
test . 

Procedures  Test .  This  measurement  instrument  is  a  hands-on  test 
designed  to  determine  the  extent  of  the  students'  facility  in  operatic 
the  6883  Test  Station.  The  test  was  developed  by  assembling  a 
pseudo-operational  procedure  for  the  test  station.  Elements  of  many 
different  technical  order  procedures  were  combined  into  a  single 
procedure  which  the  student  follows.  The  elements  were  chosen  in  a  way 
which  maximized  the  number  of  TRUs  operated,  and  required  both 
front-panel  actions  (i.e.,  setting  controls,  reading  meters,  etc.)  and 
internal  test  station  operations.  In  addition,  the  student  is  required 
to  interface  an  LRU  to  the  test  station  as  though  an  actual  checkout 
were  being  performed.  Every  effort  was  made  to  incorporate  a 
representative  sample  of  all  possible  field-type  operational  actions  for 
the  test  station. 


This  test  is  most  closely  associated  with  the  practical  portion  of 
AET  training  and  with  many  of  the  equipment  display  interactions  of  IGS 
training.  No  troubleshooting  was  required  or  intended  in  this  test.  The 
test  is  composed  of  12  distinct  segments,  each  requiring  the  student  to 
perform  a  set  of  equipment  manipulations,  and  then  to  identify  and 
explain  the  resultant  equipment  indications  (meter  readings,  visual 
discriminations,  etc.).  Of  the  55  test  items  which  comprise  the 
Procedures  Test,  19  of  them,  are  unique  to  the  6883  test  station  (i.e., 
require  equipment  manipulations  related  to  TRU  and  LRU  components  found 
only  on  the  6883  test  station).  The  remaining  36  test  items  require 
programming,  switch  setting,  meter  reading  and  other  basic  activities 
related  to  common  TRUs  such  as  DATAC ,  digital  voltmeter,  and 
oscilloscope.  A  45-minute  maximum  time  to  completion  was  set  for  this 
test  (see  Appendix  C). 

Troubleshooting  Test .  This  2-hour  pencil-and-paper  test  was 
designed  to  measure  how  well  subjects  can  troubleshoot  test  station 
malfunctions,  by  logically  analyzing  problems  and  applying  their 
understanding  of  test  station  operation  (including  signal  flow  and 
component  TRUs).  In  a  recent  report  (Cicchinelli,  Harmon,  Keller,  & 
Kottenstette ,  1980),  24  avionics  maintenance  personnel  identified  these 
skills  as  highly  relevant  to  6883  Test  Station  training  and  job 
performance.  The  2-hour  time  period  was  considered  sufficient  for 
sampling  troubleshooting  skills  and  knowledges  typical  of  the 
operational  field  environment. 

The  test  was  developed  by  DRI  staff  with  assistance  from 
experienced  ATC  instructors.  The  test  consists  of  six  separate  problems 
with  a  varying  number  of  subparts,  for  a  total  of  31  items.  During 
development  of  the  items,  care  was  taken  to  pattern  the  problems  after 
those  presented  in  the  student  workbooks  (See  Appendix  B  for  samples  of 
workbook  troubleshooting  problems.)  This  was  done  in  order  to  ensure 
that  the  problems  were  of  a  type  and  in  a  format  that  both  the  actual 
equipment  and  IGS  trainees  had  worked  during  training.  Care  was  also 
taken  to  ensure  that  none  of  the  items  duplicated  workbook  problems  or 
problems  presented  in  IGS  simulations.  Finally,  every  effort  was  made 
to  ensure  that  the  problems  did  not  include  an  explicit  troubleshooting 
methodology  taught  in  either  the  AET  or  IGS  training  regimes. 

In  place  of  the  pencil-and-paper  test,  troubleshooting  competence 
could  also  be  measured  by  a  hands-on  AET  troubleshooting  test.  This 
approach  was  not  used  for  two  reasons.  First,  it  is  desirable  to 
separate  the  physical  performance  tasks  (equipment  familiarity  and 
equipment  procedures  proficiency)  from  the  highly  cognitive 
troubleshooting  skills.  Hands-on  AET/IGS  troubleshooting  confounds 
these.  Second,  in  the  limited  time  available  for  testing  subjects,  very 
little  data  on  troubleshooting  competence  could  be  collected  from  an  AET 
hands-on  troubleshooting  test.  The  set-up  and  testing  of  six  faults  on 
the  actual  equipment  would  have  taken  up  much  more  time  than  the 
students  had  available  for  testing.  AET  troubleshooting  testing  would 
have  forced  consideration  of  fewer  troubleshooting  problems,  possibly 
only  one,  which  would  have  greatly  reduced  the  amount  of  data  collected 
and  the  test  reliability.  The  paper-and-  pencil  test  allowed  the 
testing  of  troubleshooting  reasoning  on  six  distinct  problems,  rather 
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than  just  one  or  two  available  with  an  AET  hands-on  troubleshooting 
test.  The  2-hour  maximum  time  for  completion  was  set  for  this  test  in 
order  to  stay  within  the  testing  time  requirement  provided  by  the 
school.  (See  Appendix  C-l.) 

Projected  Job  Prof iciencv  Test  (P JPT ) .  This  test  was  used  in  a 
previous  6883  simulation  evaluation  study  (Cicchinelli  et  al.,  1982). 
The  PJPT  was  designed  to  assess  the  possible  impact  of  training  on 
actual  job  performance.  Since  a  field  follow-up  of  subjects  could  not 
be  incorporated  into  the  evaluation  plan,  due  to  cost  and  time 
limitations,  this  test  was  included.  The  primary  rationale  for  this  test 
was  to  determine  whether  students  are  knowledgeable  about  basic 
electronics  tc  the  level  expected  by  field  personnel.  Development  of 
the  test  was  based  on  a  job  proficiency  criteria  analysis  which  had  been 
used  to  solicit  questions  from  field  supervisors  reflecting  the  level  of 
knowledge  expected  of  new  technicians.  From  75  items  received,  a  35 
item  pencil-and -paper  multiple-choice  test  was  developed.  The  PJPT  was 
administered  with  a  45-minute  time  limit  (See  Appendix  C-3). 

All  performance  evaluation  test  instruments  were  administered  to  a 
limited  number  of  Air  Force  students  before  initiation  of  the  evaluation 
effort.  The  full  battery  of  tests  was  administered  to  a  Block  IX  class 
(N=4)  to  identify  problems  in  test  administration  or  in  test  item 
clarity. 


Student  Sample  and  Assignment  to  Groups 


The  total  subjects  included  1  female  and  42  male  students,  of  which 
22  were  AET  trained  and  21  were  IGS  trained.  Sex  distribution  across 
treatments  was  not  an  issue  in  treatment  assignment. 


Two  early  classes  (total  number  of  students  =  5)  were  randomly 
assigned  by  class  to  each  of  the  two  treatment  conditions.  Once  it  was 
determined  that  the  anticipated  student  flow  would  be  low,  all  other 
experimental  subjects  were  assigned  to  treatment  groups  on  an  individual 
basis.  Each  entering  class  was  divided  into  two  separate  treatment 
conditions  —  AET  training  and  IGS  training  —  each  with  its  own 
instructor.  Assignments  were  made  in  proportions  required  to  maintain 
treatment  assignment  equivalency.  For  example,  an  entering  class  of  4 
students  resulted  in  2  students  each  being  randomly  assigned  to  the  AET 
and  IGS  treatment  conditions. 


No  attempt  was  made  to  match  students  to  experimental  treatments 
based  on  aptitude  test  scores  or  prior  instruction  block  scores. 
Previous  analyses  of  these  measurement  instruments  (e.g.,  Cicchinelli  et 
al.,  1982)  showed  that  performance  on  these  measures  has  only  modest 
correlation  with  performance  in  the  6883  Test  Station  block  of 
instruction.  However,  as  part  of  the  data  analysis,  scores  on  these 
measurement  instruments  and  scores  on  the  prior  block  tests  were 
analyzed  to  confirm  that  student  comparability  or  random  subject 
sampling  did  exist  in  subject  assignment  to  treatment  groups. 
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Instructor  Assignment  to  Groups 

Some  of  the  6883  Test  Station  instructors  at  Lowry  AFB  were 
assigned  to  the  ATC  Special  Projects  Office.  Personnel  in  this  office 
were  involved  in  the  IGS  project  throughout  the  design,  development,  and 
evaluation  phases.  Instructors  from  this  office  taught  the  IGS 
treatment  during  the  evaluation.  Other  ATC  6883  Test  Station 
instructors,  with  no  connection  to  the  Special  Projects  Office,  taught 
Block  IX  using  the  AET.  Therefore,  in  both  treatments,  training 
delivery  to  students  was  provided  by  instructors  highly  familiar  with 
and  partial  to  the  training  approach  used.  Two  Special  Projects  Office 
instructors  each  taught  one-half  of  the  21  IGS  students  (11  students  and 
10  students).  One  of  the  four  AET  instructors  taught  12  of  the  22 
students.  The  other  three  instructors  taught  the  remaining  10  AET 
students  (4,  4,  and  2). 

Testing  Schedule 

All  evaluation  performance  measurement  instruments  were 
administered  on  a  day  specifically  set  aside  for  this  purpose.  This  day 
immediately  followed  completion  of  the  6883  Test  Station  block  of 
instruction  (Block  IX)  and  the  ATC  end-of-block  test.  Students  were 
given  the  two  pencil-and-paper  tests  (PJPT  and  Troubleshooting  Test) 
according  to  a  schedule  that  permitted  ongoing  administration  of  the 
Prodedures  Test  to  students  on  an  individual  basis.  This  schedule  was 
approved  by  both  ATC  and  AFHRL . 

Data  Collection 

Data  collection  and  analysis  were  planned  and  managed  by  DRI 
personnel.  The  troubleshooting  and  PJPT  pencil-and-paper  tests  were 
administered  and  monitored  by  ATC  instructor  personnel,  while  DRI 
personnel  administered  the  hands-on  6883  procedures  test.  All  test 
scoring  and  data  analyses  were  performed  by  DRI  personnel.  This 
approach  ensured  consistency  of  presentation  and  scoring,  as  well  as, 
evaluation  testing  and  course  instruction  independence. 

Evaluation  Results  and  Discussion 

The  results  of  the  evaluation  are  presented  for  training 
effectiveness,  environmental  suitability,  and  cost.  The  emphasis  of  the 
evaluation  was  on  the  effects  of  differences  in  training  approaches  on 
student  performance.  This  section  focuses  on  analysis  of  student 
performance  on  the  three  primary  test  instruments. 

Training  Effectiveness 

Based  on  the  goals  of  this  study,  the  design  of  the  IGS  system,  the 
simulations  developed,  and  the  manner  in  which  the  IGS  simulations  were 
integrated  into  and  coordinated  with  the  course  and  its  POI,  several 
hypotheses  were  investigated: 

Hy  1  -  IGS  and  AET  training  result  in  equivalent  student 
performance  on  the  standard  ATC  block  test. 
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Hy  2  -  IGS  and  AET  training  result  in  students'  acquiring  an 
equivalent  ability  to  perform  operational  procedures 
on  the  6883  Test  Station. 

Hy  3  -  IGS  and  AET  training  result  in  students  acquiring  an 
equivalent  ability  to  solve  common  troubleshooting 
prob lems . 

Hy  4  -  IGS  and  AET  training  result  in  students  solving 
common  troubleshooting  problems  with  equivalent 
speed . 

Hy  3  -  IGS  and  AET  training  result  in  students  acquiring 
equivalent  job-related  experience. 

No  performance  difference  was  expected  a  priori  on  any  of  the  criterion 
measures . 

Background  and  pre-6883  training  measures .  Aptitude  tests  (ASVAB 
and  AFQT)  did  not  show  significant  differences  between  training  modes. 
This  was  not  surprising  since  students  in  the  3ABR32634A  career  field 
had  been  preselected  according  to  these  test  instruments.  Therefore, 
varability  among  students  on  these  measures  was  diminished.  (See  Table 
D-l ,  Appendix  D  for  a  summary  of  aptitude  test  results.) 

Prior  instructional  block  performance  scores  did  not  reveal 
significant  differences  between  training  modes  either.  This  was  also  to 
be  expected,  since  the  6883  Test  Station  instructional  block  is  the 
ninth  block  of  instruction  in  the  course.  Slower  students  who  might 
have  provided  performance  variability  would  likely  have  been  reassigned 
to  another  career  field  by  this  point  in  the  training  program.  In 
addition,  the  avionics  electronics  career  field  is  especially  difficult; 
therefore,  those  students  who  are  admitted  are,  overall,  students  of 
comparably  high  ability.  (See  Table  D-2,  Appendix  D  for  a  summary  of 
Block  I  -  VIII  end-of-block  test  scores.) 

Block  IX  training .  Several  measures  were  employed  to  identify 
potential  performance  differences  based  on  the  training  treatment.  The 
end-of-block  test  administered  by  ATC  personnel,  the  pencil-and-paper 
Troubleshooting  Test,  and  the  hands-on  Procedures  Test  comprised  the 
6883  training  performance  assessment  battery. 

The  end-of-block  test  results  analysis  addressed  the  question  of 
whether  students  received  comparable  training  regarding  the  6883  Test 
Station  and  its  theoretical  operation,  the  issue  addressed  in  Hypothesis 
1.  Results  of  this  test  did  not  reveal  significant  differences  between 
training  modes  (Table  5).  Mean  scores  for  AET-trained  students  (84.4) 
and  IGS-trained  students  (86.3)  did  not  differ  significantly 
[ t ( 41 ) =0 .70,  p  < .49 ] . 

The  Procedures  Test  was  developed  to  assess  student  ability  to 
manipulate  the  actual  equipment.  Analysis  of  the  results  of  this  test 
addressed  Hypothesis  2.  The  difference  in  mean  scores  for  AET  trainees 
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Table 

5. 

Evaluation  Test 

Instruments  Results 

Performance 

Training 

Mode 

Measure 

AET 

IGS 

t-Value 

Converter  Flight 

N 

22 

21 

0.70 

Control  End-of- 

X 

84.4 

86  .3 

NS 

block  Test 

S 

9.79 

8.16 

Procedures  Test 

N 

22 

21 

1.10 

Performance  Score 

X 

41  .8 

43.5 

NS 

S 

5.11 

5.30 

Procedures  Test 

N 

22 

21 

0.50 

Performance  Time 

X 

38.9 

40.8 

NS 

( in  minutes) 

S 

7.10 

15.21 

Troubleshoot ing 

N 

22 

21 

3.00 

Test  Performance 

X 

14.95 

17.76 

p<.005 

Score 

S 

3.03 

3.11 

Troubleshooting 

N 

22 

21 

0.24 

Test  Performance 

X 

82.19 

83.89 

NS 

Time (in  minutes) 

s 

20.91 

22.91 

(41.8)  and  IGS  trainees  (43.5)  revealed  no  significant  difference 
between  training  inodes  [ t ( 41 ) =1 .10,  p < . 28 ;  Table  5].  Results  of  the 
analysis  indicate  that  the  students  showed  no  detectable  difference  in 
their  ability  to  manipulate  actual  equipment,  regardless  of  training 
mode.  This  finding  was  consistent  with  the  expectation  that  AET-trained 
students  and  IGS-trained  students  would  perform  similarly  on  this 
measure.  The  implication  is  that  generic  experience  with  actual 
equipment  (e.g.,  on  prior  test  stations)  and  equipment  manipulation 
training  via  interactive  videodisc  were  sufficient  conditions  for 
students  to  gain  procedural  familiarity  with  actual  equipment. 

Time  to  completion  was  also  tracked  for  the  Procedures  Test.  Mean 
times  for  AET  students  (38.9  minutes)  and  IGS  students  (40.8  minutes) 
showed  no  significant  difference  between  training  modes  [t(39)=.50, 
p < . 6 2 ] .  Although  the  means  were  not  significantly  different,  the  IGS 
students  showed  somewhat  greater  variability  in  time  usage  than  the  AET 
students.  This  result  is  not  unexpected  since  the  AET  students  had 
consistent  exposure  to  the  equipment  during  their  practical  exercises, 
while  the  IGS  students  had  not  seen  the  physical  device  prior  to  taking 
the  test.  The  relatively  equivalent  Procedures  Test  performance,  on 
score  and  time  measures,  for  both  student  groups;  indicates  that  the 


groups  were  not  detectably  different  in  their  ability  to  operate  the 
6883  Test  Station. 

The  Troubleshooting  Test  was  developed  to  measure  the  ability  of 
students  to  diagnose  electronic  faults  in  6883  test  station  TRUs  and 
LRUs.  The  analysis  of  student  performance  on  this  measure  addressed 
Hypotheses  3  and  4. 

Analysis  of  the  Troubleshooting  Test  scores  revealed  a  significant 
difference  between  the  two  training  modes  in  solving  troubleshooting 
problems  (Table  5).  The  mean  scores  were  14.95  for  AET  students  and 
17.76  for  IGS  students.  The  significant  finding  [ t ( 41 ) =3 . 00 ,  p < . 00 5 J 
permits  rejection  of  the  hypothesis  that  the  two  training  regimes 
produce  graduates  of  equivalent  troubleshooting  ability. 

Time  to  completion  was  also  recorded  for  the  Troubleshooting  Test. 
Mean  times  for  AET  and  IGS  trained  students  (82.2  minutes  and  83.9 
minutes,  respectively)  were  not  significantly  different  [t(41)=.24, 
p < . 8 1 ] .  Therefore,  performance  of  the  AET  and  IGS  students  was  not 
detectably  different  in  terms  of  the  time  required  to  solve  the 
problems . 

Several  possible  explanations  for  the  significant  performance 
difference  in  troubleshooting  accuracy  were  considered.  First, 
troubleshooting  practice  problems  vere  available  to  IGS-trained  students 
from  two  sources:  the  student  workbooks  and  the  IG1'  system  lessons.  AET 
students  received  troubleshooting  problem  practice  from  student 
workbooks  and  diagnostic  testing  practice  during  the  practical  excerises 
at  the  end  of  Block  IX.  Still,  IGS-trained  students  could  be  perceived 
as  receiving  more  troubleshooting  practice  than  did  AET  students. 

Second  (as  described  in  Chapter  II  and  illustrated  in  Appendix  A), 
decision  making  (i.e.,  the  cognitive  component  of  troubleshooting)  and 
equipment  manipulation  continually  alternated  in  the  IGS  simulations  (as 
is  the  case  in  actual  troubleshooting  performance).  Students  get  a 
chance  to  apply  their  problem-solving  conclusions  (e.g.,  that  a  certain 
measurement  should  be  made  based  on  a  just-completed  line  of  reasoning) 
as  soon  as  they  are  reached.  Thus,  cognitive  problem  solving  and 
equipment  manipulation  mutually  reinforce  each  other,  and  do  so  in 
context.  This  may  have  yielded  a  degree  of  reality  or  relevance  to  the 
troubleshooting  problems  that  was  lacking  when  troubleshooting  problems 
were  worked  out  in  a  workbook  in  isolation  from  the  equipment,  as  was 
done  in  the  AET  training. 

Third,  the  IGS  system  was  interactive,  providing  reinforcement, 
feedback,  and  prompts.  This  means  that  the  system  required  the  active 
participation  of  the  student  in  order  to  progress  through  a  simulation. 
Active  engagement  was  enforced.  In  the  first  class  of  the  IGS 
treatment,  it  was  noticed  that  some  students  were  merely  entering  the 
decision  alternatives  (presented  in  recognition  mode),  one  after 
another,  until  they  found  the  correct  one.  This  is  a  low-effort 
strategy,  not  requiring  the  active  participation  of  the  student.  The 
IGS  software  was  then  modified  to  cause  the  terminal  bell  to  ring  when  a 
student  made  a  wrong  selection.  At  this  signal,  the  instructor  would 
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come  over  and  ask  if  the  student  needed  help.  After  this  bell  was 
installed,  the  students  applied  themselves  to  figuring  out  the  correct 
alternative  at  each  choice  point  (i.e.,  guessing  stopped).  This  had 
the  effect  of  increasing  the  students'  level  of  participation. 

Fourth,  the  IGS  treatment  employed  an  individualized  training 
approach  during  use  of  the  simulations  (vs.  the  group  training  approach 
for  AET  training).  The  individual  approach  to  IGS  system  operation  for 
problem  analysis  and  solution  may  reinforce  development  of  a 
troubleshooting  strategy  better  than,  solving  workbook  problems  through 
individual  effort  with  group  rewiew  and  critique  followed  later  by 
group  training  on  the  AET,  the  approach  used  for  AET  training. 

Beside  treatment  considerations,  test  format  and  content  were 
considered  as  possible  explanations  of  the  difference  in  troubleshooting 
mean  scores.  One  possibility  was  that,  the  problems  developed  for 
the  Troubleshooting  Test  reflected  the  types  of  practice  problems 
encountered  by  IGS  students  better  than  those  encountered  by  AET 
students.  However,  this  explanation  was  rejected  for  several  reasons: 
(a)  further  analysis  of  student  performance  on  the  Troubleshooting  Test 
showed  that  IGS  students  performed  consistently  better  on  each  of  the 
six  test  problems  than  did  the  AET  students;  (b)  the  Troubleshooting 
Test  was  developed  by  individuals,  from  both  ATC  and  DRI,  who  had  no 
familiarity  with  the  content  of  IGS  lessonware;  and  (c)  student  workbook 
problems  (see  Appendix  B)  were  found  to  be  more  similar  to 
Troubleshooting  Test  problems  (see  Appendix  C-2)  than  to  IGS  problems. 
IGS  lessons  (see  Appendix  A  for  an  example)  emphasized  the  development 
of  test  loops  and  other  tools  for  troubleshooting.  No  test  loop 
development  or  related  troubleshooting  tool  was  required  or  suggested  by 
the  Troubleshooting  Test  problems.  Therefore,  a  high  degree  of 
similarity  between  IGS  lessons  and  Troubleshooting  Test  items  was  not 
considered  to  be  the  source  of  the  difference  in  mean  troubleshooting 
scores . 

Further,  it  was  considered  possible  that,  IGS  students  were  so 
concerned  about  the  quality  of  training  available  with  the  experimental 
system  that  they  studied  the  instructional  material  to  a  greater  degree 
than  did  their  AET  student  counterparts.  In  fact,  student  comments  to 
IGS  instructors  throughout  the  study  indicated  concerns  about  their 
performance  on  the  end-of-block  test  (the  only  test  which  would  count 
toward  their  class  and  overall  averages).  If  extra  study  by  IGS 
students  did  account  for  the  Troubleshooting  Test  performance 
difference,  a  performance  difference  would  also  be  expected  on  the 
end-of-block  test.  The  results  of  that  test  have  already  been 
discussed;  no  significant  difference  was  found. 

Another  possible  explanation  of  the  difference  in  troubleshooting 
scores  was  the  distributed  nature  of  the  IGS  training,  as  opposed  to  the 
massed,  7-day  classroom/ 2-day  hands-on,  training  of  the  AET  approach. 
However,  troubleshooting  training  for  both  the  AET  and  IGS  students  was 
received  through  workbook  problems  distributed  over  7  days  for  AET  and 
9  days  for  IGS  students.  Thus,  with  respect  to  much  of  the  trouble¬ 
shooting  training,  both  treatment  groups  received  distributed  workbook 
problem  training. 
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The  significant  performance  difference  on  the  Troubleshooting  Test 
between  training  modes  is  most  likely  due  to  the  following  combination 
of  characteristics  of  the  IGS  training  approach:  troubleshooting 
practice  beyond  that  of  workbook  problems,  consistency  of  IGS  lesson 
presentation,  alternating  cognitive  planning  steps  and  videodisc-based 
equipment  manipulations,  and  delivery  of  training  on  an  interactive, 
individualized  student  basis.  These  are  characteristic,  key  features  of 
the  IGS  training  approach. 

The  Projected  Job  Proficiency  Test  (PJPT)  was  developed  to  estimate 
performance  in  a  field  environment,  based  on  equipment  familiarity  and 
basic  electronics  knowledge.  Mean  scores  on  this  test  were  18.95  and 
18.76  for  AET  and  IGS  trainees,  respectively.  This  difference  was  not 
statistically  significant  [t(40)=.14,  p < . 8 9 ] .  (See  Table  D-4,  Appendix 
D.) 


Post  Block  IX  training .  Post-6883  instructional  block  end-of-test 
scores  were  analyzed  to  identify  differences  that  may  have  resulted  from 
IGS  and  AET  training  on  subsequent  instructional  blocks.  No  significant 
findings  resulted  from  this  analysis.  (See  Table  D-4,  Appendix  D.) 

Summary  of  training  effectiveness  results .  No  significant 
differences  were  revealed  between  treatment  groups  prior  to  Block  IX,  as 
evidenced  by  aptitude  scores  and  prior  instructional  block  scores.  No 
significant  differences  were  found  between  treatment  groups  at  the  end 
of  Block  IX  for  the  end-of-block  test,  for  the  Procedures  Test,  or  for 
the  PJPT.  In  addition,  no  significant  differences  were  found  between 
the  groups  on  end-of-block  test  scores  for  blocks  after  Block  IX.  These 
results  would  imply  that  IGS  and  AET  training  provided  the  students  with 
similar  skills  as  measured  by  these  instruments.  Only  the 
pencil-and-paper  Troubleshooting  Test  showed  significant  performance 
differences  in  favor  of  IGS  training. 

Attempts  to  explain  the  difference  in  troubleshooting  accuracy 
identified  several  key  characteristics  of  the  IGS  training  approach. 
These  included  troubleshooting  practice  over  that  of  workbook  problems, 
individualized  interactive  training,  alternating  cognitive  and  equipment 
manipulation  activities  in  each  simulation,  consistency  of  problem 
presentation,  and  reinforcement  of  a  developing  troubleshooting 
strategy.  The  most  probable  explaination  is  that  this  combination  of 
factors  (not  any  one  alone)  was  the  key  to  the  difference  in 
performance.  Removing  any  one  of  these  factors  from  the  IGS  approach 
could  possibly  result  in  a  detectable  performance  loss. 

IGS  Environmental  Suitability 

This  section  of  the  report  treats  issues  related  to  hardware 
compatibility  with  the  training  environment,  maintenance  requirements, 
lessonware  authoring  capability,  and  student  acceptance  of  the  training 
system.  Instructor  acceptance  was  not  measured  in  this  effort  because 
the  two  ATC  Special  Projects  Office  instructors  had  a  developmental 
investment  in  the  IGS  system  and  were  felt  to  be  biased  in  favor  of  the 
system . 


Hardware  suitability.  The  IGS  student/author  station  rests  on  a 
table  top  and  in  a  small  cabinet.  All  components  of  the  IGS  station  are 
portable  and  easily  removed  or  replaced  as  needed.  The  station  produces 
very  little  noise  and  requires  only  a  standard  110  volt  circuit  for 
operation.  The  IGS  station  does  not  require  air  conditioning  beyond 
that  of  normal  facilities  systems. 

In  the  training  environment  that  existed  for  the  evaluation  study, 
installation  of  telephone  lines  was  necessary  to  interconnect  the  IGS 
station  with  the  AFHRL  Cyber  computer  for  system  program  execution  and 
student  record  storage.  With  the  IGS  system  in  a  stand-alone 
configuration,  the  telephone  lines  would  not  be  required. 

Maintenance .  Unscheduled  maintenance  was  minimal  on  the  IGS 
stations  during  the  evaluation  period.  Alignment  of  graphics  overlays 
with  videodisc  images  and  alignment  of  touch  points  on  the  touch  screens 
with  generated  graphics  or  videodisc  images  were  two  recurring  problems. 
However,  these  problems  were  corrected  early  in  the  evaluation  period 
through  component  repairs,  and  replacement  of  the  touch  screens  with 
touch  pads.  A  problem  endemic  to  the  training  environment  throughout 
the  study  period  was  the  random  loss  of  the  IGS-Cyber  interconnect. 
While  the  interconnect  could  be  restored  quickly  in  most  instances,  the 
disruption  to  training  was  an  irritation.  However,  student  completion 
of  IGS  lessons  was  not  significantly  affected.  The  problem  was 
determined  to  reside  in  the  Cyer  system;  therefore,  further  discussion 
of  it  is  not  included.  An  IGS  stand-alone  system  configuration  would 
eliminate  this  problem. 

Lessonware  authoring .  Due  to  the  hierarchical  structure  of  the 
on-line  task  analysis  employed  in  the  IGS,  lessonware  authoring  was 
relatively  easy  for  ATC  instructor  personnel.  All  existing  menu  items 
appeared  as  elements  which  could  be  selected  from  the  authoring  menu  at 
appropriate  levels  in  the  hierarchy.  As  additional  items  were  included 
in  the  hierarchy,  these  items  appeared  on  all  appropriate  menus.  As  a 
result,  consistency  of  lesson  structure  and  item  selection  was 
automatic.  The  existing  IGS  lessonware  was  developed  by  contractor, 
subcontractor,  and  ATC  personnel  (see  Chapter  III). 

The  issues  involved  in  establishing  the  acceptability  of  the 
developed  lessonware  were  discussed  near  the  end  of  Chapter  III. 
Initially,  lessonware  relevance  was  established  on  the  basis  of  SME 
group  consensus.  At  this  point,  lessonware  relevance  has  been 
demonstrated  by  the  Performance  Test  and  Troubleshooting  Test  results 
discussed  above.  Since  Air  Force  personnel  were  largely  responsible  for 
lessonware  development  and  since  lessonware  relevance  has  been 
established,  it  is  justifiable  to  conclude  that  the  IGS  is  useable  by 
Air  Force  ATC  instructors  for  the  development  and  presentation  of 
instructional  simulations.  Using  the  simulations  in  an  IGS  training 
regime  produces  graduates,  who  perform  similarly  on  procedural  tasks  on 
the  actual  equipment  but  perform  better  on  troubleshooting  tasks  when 
compared  to  conventionally  trained  graduates. 


Student  acceptance .  A  brief  Student  Interview  form  (see  Appendix 
C-4)  was  given  to  all  students  regardless  of  training  mode.  The  first 
page  of  the  form  used  a  5-point  scale  (where  5  corresponded  to  a  highly 
favorable  rating)  for  questions  about  6883  training  in  general.  The 
second  page  specifically  addressed  use  of  the  IGS  system  through 
open-ended  questions.  AET  students  did  not  complete  the  second  page. 
Results  of  the  Student  Interview  analysis  revealed  some  interesting 
significant  results.  Although  most  of  the  differences  in  response  were 
not  significant,  those  that  were  showed  that  the  IGS  students  perceived 
that  their  regime: 

*  included  greater  variety  of  training 
IGS  (4.4)  -  AET  (3.5) 

[  t (38)  =  3.75,  p  <  .01  ] 

*  provided  a  good  understanding  of  how  to  operate  the  equipment 
IGS  (4.0)  -  AET  (3.5) 

[  t (38)  =  2.38,  p  <  .05  ] 

*  included  more  troubleshooting  time 
IGS  (4.1)  -  AET  (3.3) 

[  t(38)  =  4.08,  p  <  .01  ] 

A  review  of  the  comments  on  the  second  page  of  the  Student 
Interview  revealed  that  students  who  used  the  IGS  equipment  found 
several  characteristics  to  their  liking.  Most  commonly  mentioned  were; 
(a)  instruction  in  developing  a  troubleshooting  approach  (9  of  24 

comments),  (b)  the  step-by-step  process  of  the  lessons  (5  of  24 

comments),  (c)  the  individual  training  format,  and  (d)  the  self-paced 
approach  (3  of  24  comments).  Favorable  comments  were  also  received 
regarding  the  use  of  videodisc  imagery,  color  enhancement  for  test 
loops,  feedback  and  prompt  messages,  and  the  fact  that  working  on  the 
system  was  fun. 

Negative  comments  focused  on  the  interconnect  problem  between  IGS 
and  the  Cyber  system  (7  of  11  comments).  Additional  negative  comments 
included  cursor  problems  (point  drift  on  touch  screens)  (2  of  11 

comments),  lessons  which  were  too  long,  and  long  warm-up  time. 

A  few  suggestions  were  offered  including  suggestions  for  expansion 
of  the  IGS  approach  to  other  test  station  instruction  blocks  and  for 
expanded  explanations  of  why  an  incorrect  answer  was  incorrect.  The 
suggestions  for  using  IGS  in  other  blocks  of  instruction  were  surprising 
in  light  of  the  students  initial  concern  about  the  quality  of 
instruction  with  the  IGS  replacing  the  AET.  Expansion  of  the  IGS 
approach  and  system  capabilities  to  other  blocks  could  improve  the 
training  effectiveness  in  content  areas  other  than  troubleshooting. 

General  comments  stressed  that  the  IGS  system  should  not  replace 
the  AET;  hands-on  time  on  the  actual  equipment  trainer  was  perceived  to 
be  an  important  part  of  training.  However,  empirical  evidence  from  the 
Procedures  Test  results  indicate  that  this  AET  perception  may  not  be 
warranted . 


Summary  of  IGS  environmental  suitability.  The  IGS  system  was 
determined  to  be  a  highly  mobile,  easily  accommodated,  low  maintenance 
system.  The  development  and  updating  of  valid  lessonware  was  found  to 
be  easily  accomplished  by  ATC  personnel.  Student  acceptance  was  found 
to  be  favorable  regarding  the  variety  of  training,  the  explanation  of 
6883  equipment  operation,  the  development  of  a  troubleshooting  strategy, 
the  amount  of  troubleshooting  practice,  the  step-by-step  instructional 
process,  and  the  individualized  operation. 

IGS/AET  Cost  Evaluation 

In  the  evaluation  of  the  IGS  and  AET  costs,  a  simple  cost  model, 
similar  to  the  one  used  in  previous  simulator  studies  (Cicchinelli  et 
al.,  1982;  Cicchinelli  et  al.,  in  press)  was  employed.  Eight  major  cost 
categories  were  considered  (Table  6).  These  included  three  types  of 
hardware  and  software  breakdowns,  plus  the  documentation  and 
administrative  costs  incurred  by  the  contractor. 

The  costs  incurred  by  the  Air  Force  Special  Projects  Office 
personnel  were  not  included.  However,  the  hours  they  spent  on 
courseware  development  were  added  into  the  total  hours  spent  and  were 
represented  in  the  development  dollars  shown  for  software. 

.Cost  figures  for  both  the  hardware  and  software  components  of  the 
IGS  system  were  based  on  contractor  records.  One  hardware  cost  item  was 
estimated:  the  central  computer.  The  computer  required  was  sized  on  the 
basis  of  the  amount  of  resources  used  in  the  AFHRL  Cyber  system.  The 
cost  was  based  on  vendor  quotes  for  a  computer  of  that  size.  No  attempt 
was  made  to  determine  the  reduction  that  might  occur  in  these  cost  as 
packaging  densities,  availability,  and  competition  increase.  The  quoted 
prices  were  simply  used  as  given.  Costs  for  the  AET  were  taken  from 
previous  6883  simulator  cost  studies  (Cicchinelli  et  al.,  1982)  and 
adjusted  to  1984  dollars  based  on  Consumer  Price  Index  figures. 

The  three  types  of  hardware  and  software  costs  were:  I  -  Off-the 
Shelf,  II  -  Standardized,  and  III  -  Unique.  For  hardware,  the 
distinction  between  Types  I  and  II  was  somewhat  arbitrary.  All  Type  I 
and  Type  II  hardware  was  off-the-shelf  equipment;  however,  Type  II 
included  modems,  multiplexers,  etc.  Only  modems  were  used  with  the  IGS 
system.  In  addition,  no  Type  III  hardware  items  were  developed  or 
purchased  for  this  effort. 

For  software,  the  operating  system  for  the  peripheral  computer  was 
the  only  off-the-shelf  item.  The  system  included  a  text  editor,  file 
manipulation  tools,  an  assembler,  and  a  high-level-language  compiler. 
For  an  operating  environment  only,  a  software  development  system  would 
not  be  required.  In  addition,  the  progrims  developed  would  not  need  to 
be  redeveloped  for  subsequent  use  in  preparing  new  courseware  and 
simulations.  Type  III  software  was  considered  to  be  the  specific 
simulations  and  courseware  developed  for  student  use  in  the  classroom. 
This  is  the  only  software  which  needs  updating  and  revision  based  on 
course  and  equipment  changes,  or  on  additional  classroom  material  as 
desired  for  expanded  training  objectives. 
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Category 


Investment  Recurring 
Cost  Cost 


Investment  Recurring 
Cost  Cos 


Type  I  Hardware 
Central  Computer 
Student  Station 

Peripheral  Computer 
with  monitor 

Video  Disc  w/  Controller 
Graphic  Overlay  w/  Cont . 
Touch  Screen  w/  Cont. 
Color  Monitor  (NTSC) 
Subtotal  -  (x2) 

Type  II  Hardware 

Modems  -  two  per  station 
Subtotal  (X2) 

Type  III  Hardware 

Hardware  Total  ( 


30,000 


4,200 

1,100 

3,000 

900 

500 

19,400 


1,500 

3,000 

0 


Type  I  Software 

Operating  System  (Dev.)  1,000 
Type  II  Software 

Id  it  or  Programs  200,000 

Instructional  Features  50,000 

Type  III  Software 

Courseware  (29  simulations  250,000 
with  text  and  graphic 
tutorials,  feedback,  and 
prompt  screens 
including  videodisc  based 
simulated  equipment  action) 
Courseware  evaluation  50,000 


Software  Total 

Document  at i on 

System  user  handbooks 
New  printings 

Administrat ive 
Cont  ract 


2,500 


200 

100 

100 

100 

100 

1,200 


52,400  $  3,700 


3  ,000 


$  551  ,000  $  3,200 


10,000 


$  2,398,900  $  244,900 


0  $  2,300 


40  ,000 


15,000 


T  o  t  a  1  s 


$  653,400  5  7,100 


$  2,413,900  $  247 ,200 


Two  major  cost  areas  were  considered  for  each  of  the  eight 
categories.  The  first  was  investment  costs  which  included  acquisition, 
installation,  and  start-up  costs.  The  second  cost  area  was  that  of 
recurring  costs:  operation,  maintenance,  updating,  and  modification  or 
revision  costs.  The  recurring  costs  shown  were  for  1  year. 

Finally,  no  cost  estimate  for  the  Centeral  Processor  and  Controls 
(CENPAC)  computer,  used  to  drive  the  6883  Test  Station,  was  included  due 
to  the  unavailability  of  cost  records.  Inclusion  of  just  the 
maintenance  costs  for  this  equipment  would  serve  to  increase  the  costs 
for  the  actual  equipment.  Thus,  the  difference  between  AET  and  IGS 
costs  is  probably  much  larger  than  indicated  by  the  current  estimate. 

Comparison  of  system  costs  between  the  IGS  and  the  actual  equipment 
shows  the  investment  cost  of  the  IGS  system  to  be  significantly  less 
than  the  initial  investment  cost  of  the  AET  ($653,400  vs.  $2,413,900, 
respectively).  Recurring  costs  for  a  full  year  were  also  significantly 
less  for  the  IGS  system  than  for  the  AET  ($7,100  vs  $247,200, 
respectively) . 

Expansion  of  these  costs  to  reflect  a  5-year  or  longer  life  cycle 
was  considered;  however,  it  seemed  unwarranted,  given  the  rate  at  which 
costs  in  this  industry  are  changing.  A  case  can  be  made  for  a 
straight-line  expansion  or  an  escalating  expansion  or  a  diminishing 
expansion  for  development  of  the  estimate.  In  short,  estimates  beyond 
the  first  year  are  far  more  tenative  and  would  vary  according  to  the 
costing  assumptions  and  the  rapidly  changing  industry  costs.  The  reader 
is  free  to  make  whatever  estimate  appears  warranted  if  projections 
beyond  the  first  year  are  desired. 


V.  STUDY  IMPLICATIONS 
Conclusions 


IGS  trained  students  performed  significantly  better  on  a  written 
troubleshooting  test  than  did  the  AET  students.  The  troubleshooting 
test  consisted  of  a  2-hour  test  of  six  problems  which  represented  t'ic 
domain  or  types  of  problems  found  in  the  operational  field.  The  mean 
troubleshooting  test  scores  were  14.95  for  the  AET-trained  students  and 
17.76  for  IGS-trained  students  [t(40)=  3.00,  p<  .005].  Differences  in 
student  performance  on  all  other  measures  were  not  significant, 
regardless  of  training  mode. 

It  is  interesting  to  note  that  the  IGS  students  (who  were  not 
exposed  to  the  6883  actual  equipment)  performed  as  well  on  the  hands-on 
equipment  procedures  test  as  did  those  who  received  their  training  on 
the  actual  6883  Test  Station.  The  mean  test  scores  were  41.8  for  AET- 
trained  students  and  43.5  for  IGS-trained  students.  However,  the 
difference  was  not  significant  [t(40)=  1.10,  p<  .28].  SMEs  designed  the 
hands-on  procedures  test  to  determine  how  well  students  could 
effectively  perform  basic  equipment  manipulations  required  to  properly 
operate  the  6883  Test  Station. 

Also  interesting  is  the  finding  that,  IGS  students  expressed 
greater  acceptability  and  perceived  a  higher  effectiveness  for  their 
training  regime  than  did  AET  students.  IGS  students  were  also  favorably 
disposed  toward  the  troubleshooting  approach  reinforced  by  the  IGS 
courseware.  This  study  suggests  that  high  task  fidelity  simulation 
(i.e.,  decision  making  combined  with  equipment  manipulation)  appears  to 
be  more  important  than  high  physical  fidelity  simulation  when 
considering  training  effectiveness  and  student  acceptance  of  a 
maintenance  trainer. 

The  IGS  system  proved  suitable  for  the  existing  Air  Force  technical 
training  environment,  providing  easy  development  of  task-based 
maintenance  lessonware.  In  addition,  the  system  required  low 
maintenance  and  operated  extremely  well  in  a  standard  classroom 
environment.  Finally,  the  IGS  system  was  substantially  less  costly  as 
compared  with  the  AET. 


Recommendations 


Based  on  the  preceeding  conclusions,  the  following  recommendations 
are  presented: 

1.  The  IGS  system  should  be  used  in  the  6883  classroom 
training  curriculum  as  long  as  an  equipment-specific 
training  approach  is  in  use. 


Use  of  the  IGS  system  to  reinforce  daily  POI  training  objectives 
has  been  shown  to  be  at  least  as  effective  as  actual  equipment  training 
for  meeting  ATC  requirements  as  evidenced  by  end-of-block  test  results. 


2.  The  IGS  system  should  be  used  in  place  of  actual 

equipment  to  provide  avionics  equipment  operation 

training . 

The  IGS  system  can  provide  equipment  operation  procedures  training 
at  least  as  well  as  training  currently  presented  on  the  actual 
equipment.  Incorporating  necessary  lessonware  and  videodisc/graphics 
image  changes,  the  IGS  system  can  meet  the  ATC  needs  for  equipment 
operation  training  for  a  wide  variety  of  different  types  of  actual 
equipment  trainers. 

3.  The  IGS  system  should  be  used  for  troubleshooting 

training  in  avionics  electronics  maintenance  training. 

The  IGS  system  and  training  approach  have  been  shown  to  deliver 
troubleshooting  training  better  than  does  AET  training.  Troubleshooting 
knowledge  and  skill  were  identified  by  field  personnel  as  critical  to 
the  avionics  maintenance  mission.  Reinforcement  of  a  consistent 
troubleshooting  approach  was  identified,  by  students  who  were  trained  on 
the  IGS  system,  as  a  desired  feature  of  the  IGS  training  approach.  The 
apparent  significant  transfer  of  training,  criticality  of  the 
maintenance  task,  and  high  user  acceptance  suggest  that  the  role  of  the 
IGS  system,  as  a  troubleshooting  trainer,  should  be  established  and 
expanded  to  other  avionics  electronics  instructional  blocks  and 
maintenance  curriculums. 

4.  The  use  of  the  IGS  system  in  courses  of  instruction 
should  be  accompanied  by  appropriate  efforts  to  convince 
instructors  (and,  therefore,  students)  of  its  value  as  a 
replacement  for  actual  equipment  in  meeting  training 
objectives . 

The  cost  effectiveness  of  the  IGS  system  has  been  demonstrated,  and 
the  system's  worth  has  been  accepted  by  the  instructors  and  students  who 
used  it.  However,  the  IGS  instructors  in  this  study  had  "ownership"  of 
the  IGS  system  through  their  involvement  with  its  design  and 
development.  A  larger-scale  institutionalization  of  the  IGS  system  will 
need  to  be  accompanied  by  appropriate  efforts  to  win  instructor 
acceptance  and,  in  turn,  student  acceptance.  Since  the  IGS  system  is 
suitable  for  use  by  Air  Force  personnel  in  lessonware  development,  an 
excellent  route  to  acceptance  should  be  through  instructor  involvement 
in  the  lessonware  development  process. 

5.  The  IGS  system  adaptive  student  learning  model  should  be 
more  fully  developed  and  tested. 

Comments  by  students  who  had  contact  with  the  IGS  system  identified 
the  step-by-step,  self-paced  approach  of  the  IGS  lesson  structure,  and 
the  feedback  and  prompt  features,  as  particularly  pleasing.  With  the 
activation  of  the  adaptive  menu  feature,  training  more  appropriate  to  a 
student's  skill  level  could  be  realized. 

6.  The  IGS  system  should  be  integrated  with  a  tutorial 
computer-based  instruction  capability. 
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The  IGS  system  provides  practice  in  applying  troubleshooting  skill 
and  theory-of-operation  knowledge  to  diagnosing  faults;  it  plays  no  role 
in  the  direct  instruction  (e.g.  tutoring)  in  a  toubleshooting  approach 
or  in  theory  of  operation.  In  the  IGS  evaluation,  the  tutorial  role  was 
played  by  the  instructor.  Although  this  may  be  optimal,  the  use  of  the 
IGS  system  to  provide  more  background  instruction  should  be 
investigated.  This  is  particularly  desirable,  since  the  existing  IGS 
hardware  and  software  are  completely  compatible  with  a  Computer  Based 
Instruction  (CBI)  system.  The  extent  of  the  potential  benefits  from  IGS 
are  as  yet  unexplored. 

7.  The  IGS  lessonware  development  editors  should  be  more 
fully  developed. 

Air  Force  personnel  did  the  bulk  of  the  lessonware  development  for 
this  project.  However,  contract  personnel  played  an  important  role  in 
the  development  of  the  initial  on-line  task  analysis  and  in  overall 
quality  assurance.  The  feasibility  of  embedding  a  default  approach  to 
electronics  troubleshooting  within  the  IGS  system  and  articulating  this 
approach  as  a  series  of  prompts  and  help  messages  to  the  developer 
should  be  investigated. 

8.  The  IGS  system  should  be  developed  into  a  stand-alone 
system. 

Numerous  complaints  by  students  who  were  trained  on  the  ISG  system 
and  the  record  of  IGS  system  maintenance  focus  on  the  fragility  of  the 
IGS/Cyber  system  interface.  A  stand-alone  IGS  system  would  correct  this 
problem. 


9.  The  use  of  the  IGS  system  as  a  performance  measurement 
device  should  be  investigated. 

It  is  very  easy  to  work  through  a  fault  isolation  exercise  with  the 
IGS  system,  an  exercise  which  incorporates  an  equipment-specific 
troubleshooting  approach  and  a  reasonable  simulation  of  equipment 
manipulation.  Utilizing  the  adaptive  model  menu  feature  of  the  IGS 
system,  personnel  being  tested  could  begin  a  simulation  running  in 
"performance  mode,"  and  the  system  would  appropriately  transition  to 
recall  and  recognition  modes  in  order  to  focus  and  direct  the  completion 
of  the  exercise,  once  errors  were  made.  A  metric  of  performance  could 
be  derived  from  the  before-test  and  after-test  changes  in  the  student 
model  over  the  course  of  a  series  of  fault  isolation  exercises. 
Moreover,  performance  measurement  could  also  be  prescriptive,  in  that 
errors  could  be  mapped  into  areas  of  the  on-line  task  analysis  needing 
further  study. 

10.  The  feasibility  of  the  IGS  system  as  a  performance  aid 
should  be  investigated. 

The  value  of  the  IGS  system  as  a  training  aid  has  been  established. 
There  are  several  characteristics  of  the  IGS  system  and  task  modelling 
approach  which  would  be  highly  adaptable  to  a  performance  aiding  role. 
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These  include  the  on-line  task  analysis  (in  which  fundamental  elements 
of  the  avionics  electronics  maintenance  task  are  included);  the  highly 
flexible  and  easily  updateable  videodisc/graphics  imagery  base;  the 
decision  menu  approach,  which,  with  the  adaptive  feature  developed, 
could  interface  with  the  user  at  the  appropriate  level  of  knowledge  and 
skill;  and  the  system  of  prompts  and  feedback  messages,  which  would  help 
the  user  refine  the  problem  approach.  The  physical  mobility  of  the  IGS 
system  and  its  limited  facilities  requirements  make  it  an  appropriate 
aid  for  the  intermediate  shop  or  even  the  flight  line  environments. 
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To  do  this  you  must  first  select  the  quarter  of  the  test 
station  which  has  the  patch  panel  In  It  —  the  upper  right 
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PROJECT  IV 


Answer  the  following  questions  pertaining  to  each  test  number  given. 
1.  Test  number  301138 

a.  On  what  A3A2A15  pins  should  the  measured  signal  be  found? 


b.  What  test  point  relay  must  be  energized  to  test  this  signal? 


c.  On  what  relay  subassembly  of  the  Switching  Complex  is  the  relay 
found? 


d.  To  which  output  pins  are  the  wipers  connected? 


e.  At  what  Feel  and  Trim  jack  and  pin  can  the  measured  signal  be 
probed?  What  should  this  value  be? 


f.  What  would  you  first  check  if  the  output  from  the  Feel  and  Trim 
was  not  within  tolerances? 


2.  Test  number  301003  displays  a  NO-GO  in  the  results  window  of 

DATAC. 


a.  What  will  be  the  first  step  in  verifying  the  failure? 


SW  G3ABR32634A  000/001  -  I  X- 12 
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b.  What  Is  the  most  probable  cause  for  this  failure? 


c.  Assume  that  the  proper  Feel  and  Trim  subassembly  has  been  removed 
and  replaced.  What  must  be  done  before  repeating  the  test  that  failed? 


d.  Test  number  301003  passes  but  test  number  301010  fails.  What  is 
the  most  probable  cause  and  how  would  it  be  checked? 


e.  Where  should  the  output  of  the  Feel  and  Trim  be  probed  with  the 
DVM  lead? 


(1)  If  the  signal  is  not  present  at  this  point,  how  would  you 
determine  if  the  failure  is  caused  by  the  test  station  or  the  LRU? 


(a)  What  in  the  test  station  would  cause  this  test  to  fail? 


(b)  What  in  the  Feel  and  Trim  could  cause  this  test  to  fail? 


(2)  If  the  signal  is  present  at  this  point,  what  should  be  checked? 


(a)  At  what  A3A2A15  pins  could  this  be  checked? 


SW  G3ABR32634A  000/ 001 - 1 X - 1 2 


(b)  How  would  this  be  checked  for  a  malfunction? 

3.  Test  number  300180  fails.  The  DVM  reads  28  VDC. 

a.  What  circuit  is  being  tested? 

b.  After  repeating  the  test  to  verify  the  failure,  what  action 
should  be  taken? 

c.  What  is  the  most  probable  component  causing  the  fail? 

d.  How  should  this  be  checked? 

PROJECT  V 

Answer  the  following  questions. 

1.  A  28  VDC  Nose  Right  Command  will  energize  which  relay  in  the  Yaw 
Trim  Circuit?  What  will  be  felt  at  J200-J? 

2.  What  causes  K8  of  the  Lower  Relay  Panel  (12C1 1 54-855 )  to 
energize?  What  will  this  do  in  the  Roll  Trim  Circuit? 


SW  G3ABR32634A  000/ 001  - 1 X - 1 2 


Answer  the  questions  following  each  given  test  configuration  for  tape 
#830701. 

1.  Test  number  000633  displays  a  N0-G0  in  the  results  window  of 

DATAC. 

a.  What  will  CENPAC  do? 

b.  What  is  the  first  step  in  verifying  the  malfunction? 

c.  What  TO  and  figure  #  would  you  use  to  physically  locate  this 
failure? 

d.  In  what  Converter  Set  area  is  this  located? 

e.  What  should  be  done  to  verify  that  the  component  you  suspect  is 
causing  the  test  failure? 

2.  Test  number  000302  displays  a  NO-GO  LO  in  the  Results  window  of 
DATAC. 

a.  What  should  be  done  irrmediately? 

b.  Which  area  of  the  power  supply  is  being  checked? 

c.  What  Converter  Set  plug  and  pin  numbers  can  be  checked? 

d.  What  should  be  measured  there? 


SW  G3ABR32634A  000/001 - 1 X-l 3 


e.  On  what  routing  diagram  sheet  number  will  you  find  the  test  loop 
diagram? 


f.  If  you  measure  the  proper  voltage  out  of  the  Converter  Set,  what 
in  the  test  station  could  also  cause  the  test  to  fail? 


g.  If  test  number  000300  passed,  would  the  answer  to  question  "f"  be 
val id? 


h.  What  should  be  checked  next?  Why? 


(1)  How  would  you  go  about  this? 


(2)  What  patch  panel  pins  should  be  checked? 


i.  If  this  is  functioning  properly,  what  is  the  next  logical 
problem? 


3.  Test  number  002342  displays  005A2A  in  the  Measurement  Window  of 

DATAC. 

a.  The  test  should  be  repeated  by  going  back  to  which  test  number? 


SW  G3ABR32634A  000/001  - 1 X-l 3 
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b.  What  is  the  name  of  the  signal  being  checked? 


c.  What  type  of  signal  is  this? 


d.  On  what  Converter  Set  jack  and  pin  number(s)  can  this  signal  be 
found? 


e.  In  what  area  of  the  C/S  is  this  signal  processed? 


f.  What  should  be  displayed  in  the  Measurement  Window  of  DATAC? 


g.  What  does  this  represent?  (Use  figure  4-4,  Table  B  of  Shop 
Systems). 


h.  What  is  the  most  probable  cause  of  the  failure? 


i.  What  other  SRUs  are  used  in  this  test? 


4.  Test  number  2880  displays  E00002  in  the  Measurement  Window  of 
DATAC. 

a.  What  test  number  should  you  request  in  order  to  repeat  this  test 


SW  G3ABR32634A  000/001  - IX- 13 


b.  What  is  the  name  and  code  of  the  signal  being  tested? 


c.  What  type  of  signal  is  being  measured?  What  should  this  be? 


d.  On  which  C/S  jack  and  pin  number(s)  can  this  signal  be  measured? 


e.  What  will  CENPAC  do  when  this  test  fails? 


f.  Test  number  9882  displays  a  GO  in  the  Result  Window  of  DATAC. 
What  does  CENPAC  do? 


g.  Test  number  9386  displays  a  PROG  STOP  in  the  PROG  STOP  Window  of 
DATAC.  GO  is  displayed  in  the  Results  window. 

(1)  What  is  the  most  probable  cause  of  the  failure? 


(2)  What  else  may  cause  the  failure? 


(v,  What  TO  and  figure  #  would  you  use  to  physically  locate 
these  SRUs? 


(4)  What  are  the  names  of  these  SRUs? 


SW  G3ABR32634A  000/001  - 1 X- 1 3 


(5)  Which  SRU  would  you  R  &  R  first? 


(6)  What  would  you  do  to  verify  a  bad  card 


(7)  What  other  SRUs  are  used  in  this  test? 


SW  G3ABR32634A  000/001  - IX- 13 


APPE  DIX  C-l 


Troubleshooting  Test  Booklet 


This  is  a  test  of  your  ability  to  troubleshoot  the  Converter/Flight  Control 
equipment.  You  will  be  presented  with  a  number  of  troubleshooting  situations  and  a  series  of 
problems  dealing  with  each  situation. 

For  each  problem,  select  the  single  best  answer  by  marking  the  appropriate  slot  with 
the  red  marker  provided.  Be  sure  to  choose  and  mark  each  answer  carefully  because  you  may 
not  change  an  answer  once  it  has  been  marked.  Also,  you  must  mark  an  answer  to  each 
problem  before  going  on  to  the  next  one. 

Mark  slot  b  below  and  then  continue  on  to  the  next  page. 


The  test  will  be  timed  so  work  as  quickly  and  accurately  as  possible.  All  necessary 
TOs  have  been  provided.  The  blank  spaces  of  this  booklet  may  be  used  as  workspaces. 

Please  enter  your  name  and  class  number  in  the  spaces  provided  below.  When  you 
have  done  so,  mark  any  one  of  the  slots  below  and  wait  until  you  are  told  to  begin  the  test. 


Name 


Class 


86 


-  'AD-A149  417  INTERACTIVE  GRAPHICS  SIMULATOR:  DESIGN  DEVELOPMENT  AND 
EFFECTIVENESS7C0ST  EVALUATION(U)  DENVER  RESEARCH  INST 
CO  H  J  PIEPER  ET  AL.  DEC  84  AFHRL-TR-84-28 
UNCLASSIFIED  F33615-81-C-0886  F7G  579 


**••******»**••*•«  situation  m********************* 


1.1  Test  Station  Maintenance  Tape  (request  number  839902)  fails  at  test  number 
202661.  The  DVM  indicates  +15.00  vdc.  Repeating  the  test  produces  the  same  results.  You 
first  step  in  diagnosing  the  problem  to  the  component  level  is  to: 


0.2 

a. 

swap  the  DVM  with  a  good  one 

a. 

b. 

turn  the  "Oscilloscope  Input" 
switch  to  DVM  to  verify  the  signal 

b. 

c. 

check  the  counter  time 

c. 

d. 

check  input  power  into  the  A3A6 

d. 

e. 

check  input/output  voltages  of 
the  FCS  Adapter 

e. 

1.2  You  have  decided  to  check  the  input/output  voltages  of  the  FCS  Adapter.  With 
a  PSM-6  you  monitor  A4A1  TP12  and  TP13  on  the  front  panel.  The  voltage  you  read  is  +15 
vdc.  This  indicates: 


1.1 

a. 

a. 

A4A1A1K13  or  associated  logic 
defective 

b. 

b. 

A4A1 A10  defective 

c. 

c. 

A4A1PS1  defective 

d. 

d. 

A4A1A1K17  or  associated  logic 
defective 

e. 

e. 

insufficient  information  at  this  point 

1.3  The  previous  problem  indicated  that  A4A1A1K17  or  associated  logic  is 
defective.  The  signal  level  of  Control  Line  B3/14  found  at  A4A131  is  0.0  vdc.  This  indicates: 


1.2 

a. 

A4A1A1K17  defective 

a. 

b. 

A4A1CR17  shorted 

b. 

c. 

A3A6A20Q6  shorted 

c. 

d. 

A3A6A20Q5  shorted 

d. 

e. 

A3A6A12  defective 

e. 

1.4  Assume  that  the  reading  in  the  previous  problem  indicated  that  A3A6A12  is 
defective.  To  verify  the  faulty  component  on  the  A12  card,  you  should: 


a.  check  outputs 

b.  check  programming  inputs 

c.  swap  the  card  with  A1 1 

d.  swap  the  card  with  A13 

e.  none  of  the  above 


1.5  Checking  the  output  at  A12  pins  AA  and  22  you  find  0.0  vdc.  What  does  this 

indicate? 

a.  A12Q13  defective 

b.  A12Q14  defective 

c.  A12Z10A  defective 

d.  need  to  troubleshoot  A20 

e.  need  to  troubleshoot  A12  relay 
driver  circuit  (Z10A,  Q13,  Ql<() 


1.6  Assume  you  are  troubleshooting  the  relay  driver  circuit.  Checking  A12Z10A  pin 
9  you  find  +5  vdc.  At  the  base  of  A12Q14  you  find  -12  vdc.  Identify  the  fault. 

a.  RI9  open 

b.  Q13  shorted  between  emitter  and 
collector 

c.  Q14  shorted  between  base  and 
collector 

d.  R20  open 

e.  R21  shorted 

This  completes  Situation  H\.  Take  your  test  booklet  to  the  test  administrator 
before  going  on.  Time  (in  min.): 
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Situation  #2  ********************* 


2.1  Feel  and  Trim  test  number  300602  fails.  The  DVM  reads  9.9999  vdc.  Repeating 
the  test  produced  the  same  results.  What  does  the  DVM  reading  indicate: 


a.  insufficient  information  at 
this  point 

b.  PSC  input  signal  overrange 

c.  PSC  output  signal  overrange 

d.  DVM  defective 

e.  A2A2  defective 


2.2  A  PSC  output  signal  overrange  was  indicated  in  the  previous  problem.  To 
isolate  this  fail  to  the  system  level,  you  should: 


a.  check  the  inputs  and  outputs  to  the 
Feel  and  Trim 

b.  check  the  output  of  A2A9 

c.  check  the  input  to  the  DVM 

d.  check  test  point  switching 

e.  check  stimulus  switching 


90 


2.3  The  output  from  the  Feel  and  Trim  at  3203  pins  R  and  T  is  115  vac.  What  could 
be  the  problem? 


a.  Feel  and  Trim  defective 


b.  A4A2  or  associated  logic 


c.  A4A2  "Mode"  switch  in  the 
"QUADRATURE"  position 

d.  A4A2  "Pre-set  Data"  switch  was 
not  depressed 

e.  adapter  is  defective 


2.4  The  reference  to  the  PSC  is  correct  and  programming  is  correct  into  the  PSC. 
What  is  the  problem? 


a.  Ratio  Transformer  defective 


b.  DVM  defective 


c.  PSC  defective 


d.  LRU  cable  defective 


|  e.  I  e.  insufficient  information  at  this  point 

This  completes  Situation  f/2.  Take  your  test  booklet  to  the  test  administrator 
before  going  on.  Time  (in  min.): 


******************  Situation  //3  ********************* 


3.1  Feel  and  Trim  checkout  test  number  300830  failed  automatic  testing.  The  DVM 
indicates  0.0  vdc.  Repeating  the  test  and  replacing  the  Upper  Relay  Panel  produces  the  same 
results.  Your  first  step  in  diagnosing  the  problem  to  the  system  level  is  to: 


2.4  ! 

a. 

swap  the  DVM  with  a  good  one 

a. 

b. 

check  the  input  voltage  to  A2A9 

b. 

c. 

R/R  stimulus  relay  12/3 

c. 

d. 

check  inputs  and  outputs  of  the 

Feel  and  Trim 

d. 

e. 

insufficient  information  at  this 
point 

e. 

3.2  The  best  first  step  in  diagnosing  the  test  failure  at  number  300830  is  to  check 
inputs  and  outputs  to  the  Feel  and  Trim.  The  input  signal  on  3234  pins  27  and  28  is  +28  vdc. 
This  indicates: 


3.1 

a. 

a. 

internal  trouble  with  the  Feel  and 
Trim 

b. 

b. 

stimulus  relay  12/4  has  welded 
•  contacts 

c. 

c. 

A3A5  defective 

d. 

d. 

A4A5  defective 

e. 

e. 

insufficient  information  at  this  point 

3.3  The  Feel  and  Trim  input  signal  on  3234  pins  27  and  2S  is  +28  vdc.  The  signal  at 
A3A2A15P14  pins  180  and  181  is  0.0  vdc.  All  cables  are  good.  What  can  you  determine  from 
this? 


3.2 

a. 

test  point  relay  1/051 
defective 

a. 

b. 

Feel  and  Trim  defective 

b. 

c. 

A3A2A31  "LRU  Power"  switch  in 
the  "OFF"  position 

c. 

d. 

stimulus  relay  12/4  defective 

d. 

e. 

insufficient  information  at  this 
point 

e. 

This  completes  Situation  #3.  Take  your  test  booklet  to  the  test  administrator 
before  going  on.  Time  (in  min.):  _ _ 


. *************  Situation  ********************* 


4.1  While  running  the  Feel  and  Trim  checkout  automatic  testing  fails  at  test 
number  300846.  The  DVM  reads  0.0  vdc.  Repeating  the  test  and  replacing  nonredundant 
board  TB4  yields  the  same  results.  Your  first  step  in  troubleshooting  the  problem  is  to  check 
the  inputs  and  outputs  of  the  Feel  and  Trim.  The  signal  level  at  3202  pin  A  is  +28  vdc.  The 
signal  level  at  3202  pin  L  is  +12  vdc.  This  indicates  a  problem  in:  U. 


3.3 

a. 

b. 

c. 

d. 


a.  the  Feel  and  Trim 

b.  stimulus  circuits 

c.  evaluation  circuits 

d.  the  DVM 


e. 


e. 


insufficient  information  at  this  time 


4.2  A  problem  in  the  evaluation  circuits  is  indicated.  Checking  A3A2A3136  pins  C 
and  A  you  find  0.0  vdc.  What  is  your  next  step? 


a.  check  test  point  switching 


b.  check  stimulus  switching 


c.  R/R  test  point  relay  1/05$ 


d.  R/R  stimulus  relay  OS/2- 


e.  R/R  stimulus  relay  09/0 


4.3  Setting  the  DVM  to  "K-OHMS,"  "TRACK,"  and  an  appropriate  range  with  a 
jumper  installed  between  A3A2A15  pins  UU26  and  VV26  you  enter  the  program 
1 10F14*203548.  The  DVM  reads  9999.9  Ohms.  What  is  your  next  step  in  isolating  the 
problem? 


a.  R/R  test  point  relay  A3A2A14K1 1 


b.  R/R  test  point  relay  A3A2A14K23 


c.  check  programming  outputs  of  A4A5 


d.  check  programming  outputs  of  A3A5 


e.  swap  DVM  with  a  good  one 
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4.4  Checking  A3A53143  and  3116  you  find  the  correct  pulses.  No  other  pulses  are 
present.  Cabling  from  A3A5  to  A3A2  is  good.  Your  next  step  is  to: 


a.  R/R  A3A5A22 

b.  R/R  A3A5A37 

c.  R/R  A3A2A14K1 1 

d.  R/R  A3A2A14K23 

e.  R/R  A3A5A72 

4.5  You  have  replaced  relay  A3A2A14K11  that  you  suspect  was  the  problem.  Now 
how  can  you  verify  that  you  have  repaired  the  problem? 


a.  check  test  point  relay  1/058  at  the 
patch  panel 

b.  reinstall  necessary  equipment  and 
repeat  test 

c.  run  Test  Station  Maintenance  Tape 
(test  request  number  839901) 

d.  resume  testing  at  the  next  zero¬ 
ending  test  number 


check  the  test  point  controller 


I 


4.6  Assume  you  check  the  relay  at  the  patch  panel  as  you  did  prior  to  replacing  it. 
That  is,  setting  the  DVM  to  "K-OHMS,"  "TRACK,"  and  an  appropriate  range  with  a  jumper 
installed  between  A3A2A15  pins  UU26  and  VV26  you  enter  the  program  110F14*20354$.  The 
DVM  reads  9999.9  Ohms.  You  should  next: 


4.5 

a. 

R/R  A3A2A14K1 1 

a. 

b. 

R/R  A3A2A14K23 

b. 

c. 

swap  the  DVM  with  a  good  one 

c. 

d. 

check  for  continuity  between  A3A2A1439 
pins  195,  196  and  37  pins  12,  13 

d. 

e. 

check  for  continuity  between  A3A2A1439 
pins  203,  204  and  37  pins  6,  7 

e. 

4.7  Using  the  P5M-6  you  read  an  open  circuit  between  A3A2A1439  pin  203  and  37  9 

pin  6,  and  a  short  between  39  pin  204  and  37  pin  7.  Since  A3A2A14K11  was  just  replaced,  - 

what  should  you  do?  \(y 


i 


a 


I 


4.6 

«*.i— • — i* 

• 

a. 

a. 

check  wiring  between  A3A2A15P14 
and  A3A2A15 

b. 

b. 

check  wiring  between  A3A2A15P19 
and  A3A2A15P20 

• 

c. 

c. 

R/R  A3A2A14K11 

ry- 

d. 

d. 

check  wiring  between  A3A2A1437,  39 
and  relay  A3A2A14K11 

• 

e. 

e. 

no  problem  is  indicated 

This  completes  Situation  <74. 

Take  your 

test  booklet  to  the  test  administrator 

• 

before  going  or. 
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Time  (in  min.):  _  _  _ 

w 

•  - 
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*•***«***••**«•**«  Situation  sV5  ********************* 


5.1  You  are  troubleshooting  Test  Station  Maintenance  Tape  (test  request  number 
839902)  test  number  202923,  which  failed  during  automatic  testing.  There  was  no  signal 
present  on  the  Oscilloscope.  You  determined  test  point  relay  1/082  is  operating  properly. 
The  signal  generator  is  also  functioning  properly.  The  signal  Control  Line  B/21  (ground)  is 
present  at  A4A131  pin  b.  Suppose  you  checked  the  output  of  A^A1A9A  and  found  the  proper 
square  wave.  U’hat  should  you  do  next? 


a.  check  the  output  of  A4A1A8A 


d.  R/R  A4A1K1 


c.  R/R  A4A1K2 


d.  R/R  A4A1CR2 


e.  none  of  the  above 


5.2  Assume  you  checked  the  input  to  A4A1A8A  and  found  0.0  vdc  (ground).  You 
should  next: 


a.  check  A4A1A9A 

b.  check  A4A1A8A 

c.  R/R  A4A1K1 

d.  R/R  A4A1K2 

e.  R/R  A4A1CR2 


5.3  Assume  the  input  to  A4A1A8A  was  the  correct  square  wave  and  the  output  from 
A4A1A8A  at  311  pin  S  was  a  constant  -28  vdc.  What  should  you  do? 


5.2 

a. 

R/R  A4A1A9A 

a. 

b. 

R/R  A4A1A8A 

b. 

c. 

R/R  A4A1K2 

c. 

d. 

check  A4A1K1 

d. 

e. 

check  A4A1CR2 

e. 

This  completes  Situation  //5.  Take  your  test  booklet  to  the  test  administrator 
before  going  on.  Time  (in  min.):  _ _ 


******************  Situation  if  6  ********************* 


6.1  When  troubleshooting  a  circuit,  you  find  that  the  input  is  present  and  the  output 
is  not  present.  What  basic  determination  can  you  make  at  this  point? 


5.3 


a. 

b. 

c. 

d. 


a.  the  circuit  is  serviceable 

b.  the  circuit  is  defective 

c.  the  input  is  being  applied  at  the 
wrong  place 

d.  the  circuit  was  improperly  designed 


e. 


e. 


insufficient  information  to  make  any 
determination 


6.2  Feel  and  Trim  test  number  300620  fails.  The  DVM  reads  0.0  vdc.  Repeating  the 
test  produces  the  same  indications.  The  first  step  in  troubleshooting  to  the  circuit  level  is  to: 


6.1 

a. 

swap  the  Feel  and  Trim  with  a  good  one 

a. 

b. 

check  the  inputs  and  outputs  of  the  Feel 
and  Trim 

b. 

c. 

swap  the  DVM  with  a  good  one 

c. 

d. 

swap  A2A9  with  a  good  one 

d. 

e. 

check  test  point  relay  1/046 

e. 

6.3  Still  at  test  number  300620,  you  have  decided  to  check  the  inputs  and  outputs  of 
the  Feel  and  Trim.  The  input  at  J2 02-d  is  0.0  vdc;  the  output  voltage  at  3234-39  and  3234-22 
is  0.0  vac.  This  indicates  a  problem  in  the: 


a.  test  station  stimulus  circuits 

b.  test  station  evaluation  circuits 

c.  Feel  and  Trim 

d.  FCS  Adapter  A4A1 

e.  stimulus  switching  circuits 


6.U  You  continue  by  troubleshooting  the  test  station  stimulus  circuits.  Checking  at 

A3A2A15  pin  X3<i  you  read  6.0  vdc.  .  cj  find  28  vdc  at  A15P6  pin  62.  What  is  your  next 

step? 

a.  R/R  A3A2A6K3 

b.  troubleshoot  A3A2A6  and  associated 
logic 

c.  R/R  A3A2A6K2 

d.  Troubleshoot  A3A2A5  and  associated 
logic 

e.  none  of  the  above 

6.5  Assume  that  in  the  previous  problem  you  have  decided  to  troubleshoot  A3A2A6 

and  associated  logic.  What  should  you  do  now  to  isolate  the  problem  to  the  relay  or 

associated  logic? 

a.  R/R  A3A2A6K3 

b.  R/R  A3A2A6K2 

c.  troubleshoot  A3A5 

d.  troubleshoot  A1A8 

e.  troubleshoot  A4A5 


6.6  When  troubleshooting  a  circuit  which  tur  .a  device  "on"  or  "off"  in  response  to 
programming  inputs,  you  find  that  the  correct  programming  is  present  to  turn  the  device 
"on."  However,  upon  closer  examination,  you  discover  that  the  o  ut  which  would  turn  the 
device  "off"  is  also  present.  The  device  being  in  the  "off"  state  is  tne  reason  you  suspected 
this  circuit  to  be  faulty.  What  is  the  problem? 

a.  programming 

b.  the  "on"  circuitry 

c.  the  "off"  circuitry 

d.  the  device 

e.  some  other  circuit 

6.7  You  are  troubleshooting  A4A5.  Checking  test  points  for  steering  numbers  1,  2, 
3,  and  5  you  find  that  the  signals  are  correct.  At  test  point  3126  you  find  a  constant  logic 
high.  What  action  should  you  take? 


6.6 

a. 

a. 

R/R  A3A2A6K3 

b. 

b. 

R/R  A4A5A57 

c. 

c. 

check  the  input(s)  to  A4A5A56 

d. 

d. 

check  the  input(s)  to  A4A5A 57 

e. 

e. 

R/R  A4A5A56 

6.8  The  input  at  AUA5A56  pin  A  is  a  logic  low.  What  should  you  do  next? 


6.7 

a. 

R/R  A3A2A6K3 

a. 

b. 

R/R  A4A5A57 

b. 

c. 

check  inputs  to  A4A5A57 

c. 

d. 

R/R  A4A5A56 

d. 

e. 

R/R  A<tA5A35 

e. 

The  test  is  now  completed.  Please  return  your  test  booklet  to  the  test 
administrator. 


6.8 

a. 

b. 

c. 

d. 

e. 


Time  (in  min.): 


Perform  the  following  steps: 

a.  On  power  distribution  panel  A1A1 ,  position  400  CPS,  60  CPS 
MAIN,  and  60  CPS  OUTLETS  circuit  breakers  to  ON  Cup). 

b.  On  power  distribution  panel  (A1A1),  rotate  SELECTOR  switch  to 
400  CPS  PHASE  A  while  observing  that  MAIN  VOLTAGE  meter 
indicates  115  (+.  ID  volts. 

c.  On  variable  power  control  (A1A5) ,  rotate  PHASE  SELECTOR  switch 

to  C  and  adjust  PHASE  C  ADJUST  control  to  obtain  an  indication 
of  115  volts  on  voltmeter.  Observe  an  indicatin  of  400  (+20) 

Hz  on  frequency  meter. 

d.  On  ratio  input  filter  (A2A2) ,  rotate  FILTER  MODE  switch  to 
INVERTED  and  observe  that  INVERTED  indicator  lamp  lights  and 
NON-INV  indicator  lamp  goes  out.  Return  FILTER  MODE  switch  to 
PROGRAM. 


Connect  the  Feel  and  Trim  provided  for  check-out  as  follows: 

a.  Connect  Feel  and  Trim  adapter  (A44583)  to  patch  panel 
(A3A2A15). 

b.  Connect  P4  of  cable  assembly  W62  to  J1  on  adapter. 

c.  Connect  P4  of  cable  assembly  W63  to  J2  on  adapter. 


v  pill  TnV  <f>  ««  V  .<•  ■  -  I  ' 


3.  Assume  that  all  test  station  preparation  and  turn-on  is  complete.  Initiate 

UUT  Feel  and  Trim  (part  no.  120154-869)  testing  as  follows: 

a.  On  DATAC  drawer  (A1A3)  rotate  MODE  switch  to  NORMAL. 

b.  On  DATAC,  enter  applicable  test  request  number  from  figure  below  and 
depress  TEST  REQUEST  pushbutton. 

c.  Verify  that  TEST  NUMBER  display  on  DATAC  agrees  with  test  request 
number  in  figure. 

d.  Depress  START  pushbutton  and  verify  that  date  in  MEASUREMENT 
display  agrees  with  date  in  figure. 

e.  Depress  START  pushbutton  to  begin  testing. 


Program  ' 

Pape 

Test 

Station 

Test 

Request 

Number 

Section 

Number 

Date 

And  Display 

5A9-2-42-28-1CT-1 

29  September 
1976  (290976) 

12A6883-1  and 
-2 

120154-855 

120154-867 

120154-869 

830401 

II 

81 F-12C060-U033- 
OOA-OOl 

9  December 
1980  (091280) 

120154-887 

120154-889 

830402 

III 

4.  At  this  point,  interrupt  the  automatic  test  sequence  and  jump  to 
test  #301740.  Re-initiate  automatic  testing  at  that  point. 


5.  Automatic  testing  stops  at  #301752.  Perform  the  following  steps: 


a.  On  test  panel  (A3A2A31)  rotate  OSCOP  INPUT  SIGNAL  SEL  sw  to 
DIGITAL  MULTR. 


b.  Using  the  DIFFERENTIAL/DC  OFFSET  AMPLIFIER  in  the  oscilloscope, 
set  the  oscilloscope  controls  as  follows: 


-DC  or  A  coupling  sw 

+DC  or  B  coupling  sw 

VERNIER 

V/DIV 

SWEEP  MODE 


Depress  in  or  set  switch  to  DC 
Depress  in  or  set  switch  to  DC 
Fully  cw 
.5 

AUTO 


c.  Interpret  the  oscilloscope  signal  and  report  your  findings  to 
the  test  administrator. 


6.  The  lack  of  an  oscilloscope  signal  in  the  previous  step  indicates  a 
failure  at  test  #301752.  Reseat  the  patch  panel  adapter.  Then 
repeat  the  test  using  one  of  the  two  acceptable  methods  for 
repeating  tests.  Appropriate  TO  information  is  located  at  the  end 
of  this  test. 


7.  There  is  still  no  oscilloscope  signal.  Use  an  alternative  method 
from  the  one  used  in  step  6  to  repeat  test  #301752. 

8.  Verify  that  the  lack  of  a  signal  is  not  due  to  a  problem  with 
oscilloscope  channel  A  by  rerouting  the  input  to  channel  A  through 
channel  B.  Report  your  findings  to  the  test  administrator. 

9.  Verify  whether  or  not  the  signal  applied  is  getting  to  the  Feel  and 
Trim.  Perform  the  necessary  steps  to  check  for  continuity  using  a 
DVM  probe  and  patch  panel  pins  V20  and  Z20  and  report  your  findings 
to  the  test  administrator. 

Caution:  If  you  choose  to  disconnect  cable  A2A2 ,  handle  carefully; 

pins  on  the  A2A2  cable  are  "hot." 

10.  Program  Relay  04/0  using  the  following  code: 

131024*4051 

Report  whether  or  not  the  relay  is  operating  to  the  test 
administrator . 
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11.  Perform  a  visual  inspection  of  the  Signal  Generator  (A4A4) .  Report 
your  results  to  the  test  administrator  before  t ak ing  any  action. 

12.  Perform  the  necessary  steps  to  remove  and  replace  board  A25  in  the 
Switching  Control  Unit  (A3A6) .  This  board  is  accessible  from  the 
underside  of  the  drawer  and  all  screws  have  been  loosened  to  save 
time . 
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PROCEDURAL  TEST:  SET-UP  AND  MATERIALS  LIST 


I 

I 


* 

4 


i 

I 


1.  Student  Test  Booklet 

2.  Administration/Scoring  Guides 

3.  Stop  Watch 

4.  TOs: 

a.  33D7-17-15-2 

b.  5A9-2-42-2S-1 
C.  33D7-42-1-132 

5.  6883  test  station  set  for  normal  LRU  testing  except  as  follows: 

a.  Blown  fuse  in  signal  generator 

b.  Power  off 

c.  O-scope:  +and  -  AC  power  in,  SWEEP  MODE  not  AUTO,  V/DVI  at  .05, 
VERNIER  fully  CCW,  channel  A  &  B  cables  at  normal 

d.  SIGNAL  SELECT  5W  at  AUTO  TEST  POINT 

e.  DVM  to  allow  reset  via  page  9 

f.  FILTER  MODE  on  A2A2  to  NON-INV 

g.  A1A1  PHASE  SELECTOR  to  C 

h.  A 1A5  PHASE  SELECTOR  to  A 

6.  Additional  equipment: 

a.  Feel  and  trim  (12C1 154-869)  with  cover  removed  and  cables  attached 

b.  Feel  and  trim  adapter  (A44583-001)  in  drawer  but  not  secured 

c.  Patch  board 

d.  DVM  cable  in  tool  kit 

e.  Spare  fuse  for  signal  generator  in  place 

f.  Spare  -28V  relay  driver  (A33613-001)  for  replacement  in  A3A6  TRU 
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PROCEDURAL  TEST  Name 

ADMINISTRATION/SCORING 

GUIDE  Class 

1 

1-6 

7-12 

1 

Training  Code 

13-14 

kead  to  the  student: 

■ 

4 

B 

"This  is  a  test  o f  how  well  you  can  perform  the  procedures 
normally  involved  in  the  operation  and  maintenance  of  the  6883 
test  station.  While  the  actions  you  will  be  asked  to  complete  are 
typical  of  those  encountered  in  the  field,  they  are  not  meant  to 
represent  any  specific  set  of  operation  or  maintenance  procedures. 
You  are  asked  to  simply  complete  the  steps  specified  in  the  testing 
booklet  as  quickly  and  accurately  as  possible.  Complete  only  those 
steps  specified.  TOs  are  available  for  reference  if  necessary,  and 
all  appropriate  equipment  has  been  provided.  Are  these  any 
questions?" 

1 

Answer  questions  then 

"begin." 

i 

Start  clock. 

•. 

Pg-  1 

Note  errors/comments 

1. 

Remove  all  jewelry 

— 

15 

i 

2. 

Power  switches  "on"  in  proper  sequence 
(L  to  R) 

16 

* 

3. 

Rotate  SELECTOR  switch  to  400  CPS 
PHASE  A;  observe  U5v 

17 

B 

4. 

Rotate  to  PHASE  C 

18 

3. 

Adjust  PHASE  C  control  to  show  115v 
and  400  Hz 

19 

B 

6. 

Rotate  FILTER  MODE  to  INVERTED; 
observe  lamps 

20 

7. 

Return  FILTER  MODE  to  PROGRAM 

21 

B 

1 _ _ _ 

Note  Time:  _  _ / _  __ 

Reset  clock 

22-25 

;!• 

PTASG-1 

2/22/83 

B 
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Note  errors/comments 


8.  Power  switch  on  test  panel 
(400  CPS,  A3A3)  to  OFF 

9.  Select  and  install  adapter 
(Caution:  Be  sure  that  power 
is  off) 

10.  Connect  cables  to  adapter,  turn 
power  back  on 

Note  time:  _  _ / 

Reset  clock 


11  Set  MODE  switch  to  NORMAL 

12.  Enter  and  request  test  #830401 

13.  Verify  test  number,  press  START 
(allow  1  minute) 

14.  Verify  test  date,  press  START 
(allow  1  minute) 

Note  time:  _  _ / 

Reset  clock 


15.  Turn  MODE  switch  CW  through  STOP, 
back  to  NORMAL  (or)  turn  MODE  switch 
CCW  to  SINGLE  TEST  back  to  NORMAL 

16.  Enter  test  #301740,  TEST  REQUEST 

17.  Press  START 
(allow  1  minute) 

Note  time:  _  _ / _ 

Reset  clock 


Note  errors/comments 


18.  Rotate  OSCOP  INPUT  SIGNAL  SEL 
switch  to  DIGITAL  MULTR 

19.  Press  NEG  DC  coupling  switch 
Press  POS  DC  coupling  switch 
(or  slide  coupling  switches) 

20.  Set  VERNIER  to  fully  CW 

21.  Set  V/DIV  at  .5 

22.  Set  SWEEP  mode  on  AUTO 

Allow  5  minutes  for  O-scope  setup. 

23.  Student  says  "no  signal" 


Note  time: 

Tell  student  to  continue 
Reset  clock 

_ /__ 

24.  Reseat  adapter  (remove 

right  hand 

cable  if  necessary) 

25.  Rotate  MODE  switch  to  REPEAT,  press 

START 

Note  time: 

_ /___ 

Reset  clock 

26.  Rotate  MODE  switch  to  NORMAL 

27.  Using  TO  page  in  booklet,  locate 
last  zero-ending  test  # 

28.  Enter  Test  #301740,  TEST  REQUEST 
and  START.  Allow  3  minutes. 

Note  time:  _  _ / 

Reset  clock 


Note  errors/comments 


29.  Disconnect  A  6c  B 

30.  Reconnect  A  to  B  input 

31.  Select  channel  B  on  DISPLAY  switch 
(c  depress  appropriate  pushbuttons) 

Student  reports  "still  nothing" 

Note  time;  _  _ / 

Tell  student  to  continue 
Reset  clock 


32.  Power  switch  on  test  panel  to  OFF 
(Caution:  Power  must  be  off  before 
continuing) 

33.  Remove  adapter,  install  patch  board 
(optional:  turn  power  back  on) 

Set  up  DVM  as  follows: 

34.  POWER  to  FILTER  OUT  or  any  setting 
other  than  REMOTE 

35.  PRINT  CONTROL  to  TRACK 

36.  FUNCTION  to  K  ohms 

37.  RANGE  to  K  ohms 

38.  Remove  A2A2  input  cable; 

Connect  DVM  cable 

39.  Student  has  following  options. 

Check  appropriate  lines.  No 
other  options  are  acceptable. 

39. a.  Check  Pin  ii s  V20-Z20, 
reading  should  be  0  (or) 

39. b.  Set  MODE  switch  to  KEYBOARD 
(or)  reading  should  be  all  9s. 

39.c.  Set  MODE  switch  to  KEYBOARD 
Enter  #131024,  then  ADDRESS 
Enter  4051  then  INFO 

Note  time:  _  _ /  _ 

Tell  student  to  continue 
Reset  clock 


Note  errors/comments 

40.  Set  MODE  switch  to  KEYBOARD 
(already  set  if  student  used 
options  39. b.,  39.c.,  above) 

41.  Enter  #131024  then  ADDRESS 
(optional  if  student  used  option 
39.c.,  above) 

42.  Enter  4051  then  INFO  (or) 

Enter  3051,  then  INFO  (if 
option  39.C.  used  above) 

43.  Student  reports  that  relay  is  operating 

Note  time:  _  _ / 

Tel!  student  to  continue 
Reset  clock 


44.  Locate  Signal  Generator  (A4A4) 

45.  Detect  blown  fuse 

No*e  time:  _  _ /  _ 

Tell  student  to  "remove  power  from 
the  signal  generator  then  replace 
the  fuse" 

Reset  clock 

46.  A4A4  power  to  OFF  (Caution:  Does  not 
remove  power  to  fuse.  Tell  student 
"before  continuing  be  aware  that  the 
fuse  is  wired  before  the  switch  and  is 
therefore  still  hot.") 

47.  Replace  fuse  via  fuse  cap  using 
signal  generator  spare  (mark  as 
incorrect  if  student  turns  off  power 
to  test  station  unless  fuse  does 

not  come  out  with  fuse  cap;  if 
latter  case,  power  off  is  correct) 

48.  Power  to  ON;  fuse  light  is  now  off 


Note  time: 

Tell  student  to  continue 
Reset  clock 


/ 


Note  errors/comments 


49.  Enter  14,  ADDRESS 

50.  Enter  15,  ADDRESS 

(Caution:  Do  not  allow  student  to 
proceed  unles  steps  49  and  50 
performed.) 

51.  Locate  SCU 

52.  Turn  power  OFF  (Caution:  Do  not 
allow  student  to  extend  drawer  unless 
power  is  off.) 

53.  Extend  unit  (lower  cover  removed) 

54.  Locate  and  remove  component  (A25) 

55.  Replace  unit 

Note  time:  ./ 


Debrief  student,  answer  questions.  Reset 
test  station. 

Instructor  rating  of  student  ability 
(1  =  very  poor,  7  =  excellent) 


INSTRUCTIONS : 

Tor  each  of  the  35  questions  in  this  test,  select  your  best 
answer  by  circling  the  appropriate  letter  on  the  answer  sheet.  Do 
not  write  in  the  test  booklet. 

Be  sure  to  mark  only  one  answer  per  question.  If  you  are  unsure 
about  an  answer,  circle  your  best  guess.  Try  to  answer  all  of  the 
questions. 

You  will  have  45  minutes  to  complete  this  test. 

Do  not  turn  the  page  until  you  are  told  to  begin. 


to 


* 


> 


1.  The  converter  set  processes  what  type  of  data  from  the  computers 
provide  analog  outputs? 

A.  Analog-Digital  C.  Serial  Digital  Data 

B.  Discrete  Data  D.  Parallel  Digital  Data 

2.  The  amount  of  times  we  do  a  given  signal  is  know  as  the: 

A.  Command  Pulse  C.  Data  Request 

B.  Rate  Group  D.  Acknowledge 

3.  Which  area  of  the  converter  controls  the  TOC? 


A.  Area  I 

B.  Area  I  and  III 

A.  The  arithmatic  section  of  a 

A.  Central  Processing  Unit 

B.  Core 


C.  Area  III 

D.  Area  II 

computer  is: 

C.  Input/Output 

D.  None  of  the  above 


5.  The  purpose  of  the  05  tape  is  to  check: 

A.  Inputs  C.  Outputs 

B.  Bite  D.  Converter  Set  Overloads 

6.  How  many  outer  limit  checks  are  you  allowed  to  fail  on  any  test? 

A.  None  C.  Two 

B.  One  D.  Three 


7.  AJLA2  (Power  Supply  Controller)  controls  the  power  out  of  which  drawer? 

A.  Variable  Power  Control  (A1A5)  C.  Power  Supply  (A3PS1) 

B.  Power  Supply  (A2A9)  D.  Micrologic  Power  Supply  (A1A9) 

8.  Switching  Control  Unit  (A3A6)  controls  the  setting  of  routing  relays 
inside  the  following  drawers: 


A  • 

FCS  Adapter  (AAAI) 

C. 

Ratio  Transformer 

(A4A6) 

B. 

Transformer  Converter  (A4A2) 

D. 

All  the  above 

The 

VD.  in  Stimulus  Relav  Can  controls 

H 

tha: 

rt  * 

Setting  of  a  relay  supplying 
+28  vdc  to  the  can 

C. 

Information  Input 

lines 

B. 

Setting  of  relay  supplying 
+12  vdc  to  the  can. 

D. 

Steering  Input  lines 

I  1  H 


10.  A  Flip  Flop  needs  what  signals  to  set: 


13, 


A. 

A  low  on  1,  high  on  0, 

C. 

A  high  on  1,  low  on  0, 

and  a  clock  pulse 

and  a  clock  pulse 

B. 

A  high  on  1,  high  on  0, 
and  no  clock  pulse 

D. 

None  of  the  above 

The 

Emergency  Off  button  is  located: 

A. 

On  Cenpac 

C. 

On  the  Stimulus  Controller 

B. 

On  DATAC 

D. 

On  the  Test  Point 

Control! er 

The 

Programming  131023*  34353651Info . 

refers  to: 

A. 

Resetting  of  Stimulus  Relay 

C. 

Setting  of  Relays 

034, 

035,  036 

034,  035,  036 

D. 

Setting  of  Relays 

102, 

343,  536 

B. 

Setting  of  Relays  034 

A4A6  provides  Inputs  (during  maintenance  testing)  to: 

A. 

DATAC 

C. 

A4A2 

B. 

A4A7 

D. 

Cenpac 

A  F02A250V4A  fuse  is: 

A. 

A  slow  Blow  250  Volt  4  Amp 

C. 

Both  of  the  above 

B. 

A  fast  Blow  250  Volt  4  Amp 

D. 

None  of  the  above 

Is 

it  permissible  to  replace  a 

fast  Blow  fuse  with  a  slow  Blow? 

A. 

Yes 

C. 

Sometimes 

B.  No 


D.  Depends  on  the  amperage 


16. 


Is  it  permissible  to  replace  a  fuse  of  a  given  amperage  with  one  of  a 
higher  amperage? 


A.  Sometimes 

B.  Yes 


C.  No 

D.  Depends  on  the  voltage 


17.  The  400  CPS  on-off  switch  on  A3A2A31  is  used  to  control: 


A.  Signals  applied  to  the  LRU 

B.  Signals  applied  within  the 
station 


C.  Neither 

D.  Power  (as  the  STA,  power  off  switch) 


18.  The  DC  volts  meter  on  A1A2  measures  the  output  of  which  power  supply? 


A.  A2A5  Digital  Interface  Unit  C.  A2A3  Serial  Digital  Adapter 

B.  Parallel  Digital  Adapter  A2A4  D.  None  of  the  above 


19.  Pover  Stimulus  Relays  can  be  tested  at  the  patch  panel: 


A.  By  jump  2  sets  of  contacts 
together 

B.  They  cannot 

20.  Stimulus  Relays  can  be  tested  at 

A.  Programming  the  Relay  and 
ohms  checking  between  the 
normally  open  contacts  and 
the  wiper 

B.  They  cannot  be  checked 

21.  A  coil  will  usually  ohms  check: 

A.  As  an  open 

B.  As  one  or  two  ohms 

22.  A  rectifier  is  used  to: 

A.  Change  AC  to  DC 

B.  Change  DC  to  AC 

23.  A  circuit  breaker  that  will  not 


C.  In  the  usual  manner 

D.  By  programming  a  Test  Point  Relay 
the  patch  panel  by: 

C.  Programming  the  Relay  and  ohms 
checking  between  the  normally 
closed  contacts  and  the  wiper 

D .  A  or  C 


C.  A  very  high  number  of  ohms 

D.  It  cannot  be  ohms  checked 


C.  Balance  the  line 

D.  None  of  the  above 

let  usually  indicates: 


A.  A  short  in  the  circuit  C.  A  shorted  filter  capacitor 

B.  A  broken  circuit  breaker  D.  All  or  any  of  the  above 


24.  If  you  are  not  sure  about  a  problem,  vou  should: 


A.  Look  at  the  T.O.  and  fake  it 

B.  Ask  someone 

25.  A  pulse  period  is  measured  by: 

A.  An  o-scope 

B.  Counter  Timer 

26.  The  front  panel  knobs  of  the  TRUs 
Switch  in  which  position? 

A.  Normal 

B.  Stop 


C.  Push  start  on  DATAC  and  proceed 

carefully  , 

D.  Go  for  a  coffee  break 


C .  DATAC 

D.  A  or  B 

can  be  enabled  by  the  DATAC  Node 


C. -  Manual 

D.  Keyboard 


27.  The  clock  input  for  A2A1,  A2A3,  A2A4  comes  from: 


A.  DATAC 

B.  Counter  Timer 


C.  Signal  generator 

D.  None  of  the  above 


An  absence  of  a  polarity  sign  on  the  DVM  indicates: 


A. 

No  115  AC 

C. 

Counter  Time  Circuit  Breaker  on 
Micrologic  Power  Supply  tripped 

B. 

No  drive  from  DATAC 

D. 

Cenpac  off  line 

The 

Test  Station  Interconnection  Diagram  is  used  to: 

A. 

Get  between  cards  inside  a 
drawer 

C. 

Locate  pins  on  unlisted  cables 

B. 

Get  between  drawers 

D. 

It  is  not  used 

A  3 

Input  OR  gate  requires  what 

Inputs 

for  a  high  output? 

A. 

All  highs 

C. 

2  highs  and  1  low 

B. 

All  lows 

D. 

A  and  C 

26YAC  REF  voltage  on  A3A2A32  is 

used  in 

: 

A. 

CSS  (A2A8) 

C. 

Programmable  Transformer/ 
Converter  A4A21 

B. 

Ratio-Transformers  A4A6  and 
A4A7 

D. 

All  of  the  above 

To 

forward  bias  a  PNP,  you  need 

to  have: 

A. 

A  more  negative  on  the  base 

C. 

A  more  negative  on  the  base  than 

than  the  emitter 

the  collector 

B. 

A  more  positive  on  the  base 
than  the  emitter 

D. 

A  or  C 

What  area  of  the  converter  controls  the  outputs? 

A. 

Area  I 

C. 

Area  III 

B. 

Area  II 

p. 

Both  A  anc  C 

The 

purpose  of  the  03  tape  is  to  check 

A. 

Inputs 

C. 

Outputs 

B. 

Bite 

D. 

Converter  Set  Overloads 

if  : 

you  are  in  Redundant  Ladder 

of  the  converter,  you  can  check  to  be 

sure  by  using  a  probe  and  checking  J609  Pin  64.  It  vill  be  at: 


A.  5  volts 

B.  10  volts 


C.  Zero  volts 

D.  28  volts 


STUDENT  INTERVIEW 


CLASS  # 


DATE 


For  each  of  the  questions  below,  please  circle  the  number  that  best 
expresses  your  opinion  on  a  scale  from  1  to  5,  where  1  =  Not  at  all,  and 
5  =  Very  muc^. 


1.  Did  you  have  sufficient 
time  on  the  equipment  in  the 
6883  block? 

2.  Was  your  time  on  the  equipment 
well  utilized? 


Not  at  All  Verv  Much 

1  2  3  4  5 

1  2  3  4  5 


3.  Was  there  variety  in  the  training 
on  the  equipment? 

4.  Do  you  have  a  good  understanding 
of  how  to  operate  the  equipment? 

5.  Do  you  feel  comfortable  operating 
the  equipment? 

6.  To  what  extent  did  equipment 
malfunctions  hinder  your  training? 

7.  To  what  extent  did  equipment 
malfunctions  benefit  your  training? 

8.  How  much  of  your  training  time  on 
the  equipment  was  spent  on 
trouble-shooting? 

9.  Would  you  like  to  have  had  more 
trouble-shooting  experience? 


1  2  3  4  5 

2  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 


10.  Do  you  feel  that  you  have  had  12345 

adequate  trouble-shooting  experience 
for  your  field  assignment? 


11.  Was  your  training  instructor  helpful 
in  explaining  the  equipment  and 
its  use? 


1 


2 


3 


4 


5 


12.  What  were  your  impressions  of  the  GS  simulator  as  a  training  device? 
Positive?  Negative? 


13.  Do  you  feel  that  the  GS  is  sophisticated  enough  for  6883  training? 
Is  it  too  sophisticated? 


14.  What  do  vou  see  as  the  simulator's  good  points?  What  things  are  especially 
helpful  in  terms  of  training? 


15.  What  do  you  see  as  the ' simulator' s  bad  points?  Are  there  any  things  about 
the  equipment  that  you  feel  hindered  training? 


16.  Are  there  anv  things  about  the  simulator  or  the  way  it  was  used  in  training 
that  you  think  should  be  changed? 


Anv  other  comments? 


■ 

1  ' 

Table  D-l 

• 

ri 

Aptitude  Test  Scores 

• 

Test 

Training 

Mode 

i 

AET 

IGS 

t-Value 

• 

ASVAB  Composite 

General 

N 

22 

21 

0.98 

X 

67.05 

70.95 

NS 

S 

14.69 

11.14 

* 

Mechanics  1 

N 

22 

21 

0.65 

• 

X 

62.27 

65.71 

NS 

s 

17.58 

17 .27 

Administrative 

N 

22 

21 

-0.54 

• 

X 

67.05 

64.29 

NS 

• 

S 

12.88 

19.83 

Electronics 

N 

22 

21 

0.82 

X 

72.73 

74.76 

NS 

S 

7.52 

8.73 

s 

AFQT 

N 

22 

21 

1  .64 

• 

X 

65.59 

72.81 

NS 

*  ■ 

s 

15.87 

12.81 

,  \ 

Table  D-2 


Table  D-3 

Block  Test  Scores  After  Block  9 


Track 

Block  of  Instruction 

Training  Mode 

I  II 

AET 

IGS 

t-Value 

—  10 

Computer 

N 

16 

16 

0.93 

Test  Station 

X 

93.44 

95.63 

NS 

s 

7.47 

5.74 

—  11 

Attitude  and  Rate 

N 

16 

16 

0.33 

Test  Station 

X 

88.75 

89.69 

NS 

s 

8.27 

7.63 

10  — 

Displays 

N 

6 

5 

0.84 

Test  Station 

X 

89.00 

92.00 

NS 

S 

7.48 

2.83 

11  12 

Video 

N 

18 

18 

0.99 

Test  Station 

X 

81.39 

83.78 

NS 

S 

6.78 

7.62 

12  13 

Receiver  Transmitter 

N 

18 

18 

0.42 

&  Modulator 

X 

81 .44 

82.85 

NS 

Test  Station 

S 

9.74 

10.95 

Note : 

Sample  size  differences  are  due  to  the 

fact  that  two 

training 

tracks 

(I  and  II)  are  available 

after  block  9 

,  with  everyone  being 

required  to  take  Video  Test  i 
Test  Station. 

Station 

and  Receiver 

Transmitter  & 

Modulator 

Table  D-4 

Projected  Job  Proficiency  Test 

Test 

Training  Mode 

AET  IGS 

t -Value 

Projected  Job 

N 

21  21 

0.14 

Proficiency  Test 

X 

18.95  18.76 

NS 

8 

3.28  5.39 
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